-- Logs begin at Tue 2022-09-27 09:07:44 UTC, end at Tue 2022-09-27 11:34:23 UTC. -- Sep 27 09:46:49.946296 ncdv-master-1 systemd[1]: Started Devstack devstack@c-vol.service. Sep 27 09:46:53.073419 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "singleton_lock" {{(pid=138302) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 09:46:53.074359 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=138302) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 09:46:53.074359 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=138302) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 09:46:53.512807 ncdv-master-1 cinder-volume[138302]: INFO dbcounter [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Registered counter for database cinder Sep 27 09:46:53.532933 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_db.sqlalchemy.engines [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138302) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:46:53.541434 ncdv-master-1 cinder-volume[138302]: DEBUG dbcounter [-] [138302] Writer thread running {{(pid=138302) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:46:53.867861 ncdv-master-1 cinder-volume[138302]: INFO cinder.rpc [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version. Sep 27 09:46:53.873639 ncdv-master-1 cinder-volume[138302]: INFO cinder.rpc [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version. Sep 27 09:46:53.883654 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_db.sqlalchemy.engines [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138302) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:46:53.961547 ncdv-master-1 cinder-volume[138302]: INFO cinder.volume.manager [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Determined volume DB was empty at startup. Sep 27 09:46:53.962169 ncdv-master-1 cinder-volume[138302]: DEBUG cinder.volume.manager [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Cinder Volume DB check: vol_db_empty=True {{(pid=138302) __init__ /opt/stack/cinder/cinder/volume/manager.py:290}} Sep 27 09:46:53.974598 ncdv-master-1 cinder-volume[138302]: INFO cinder.volume.manager [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Service not found for updating active_backend_id, assuming default for driver init. Sep 27 09:46:53.975423 ncdv-master-1 cinder-volume[138302]: WARNING stevedore.named [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager Sep 27 09:46:53.975872 ncdv-master-1 cinder-volume[138302]: WARNING castellan.key_manager [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' Sep 27 09:46:53.977080 ncdv-master-1 cinder-volume[138302]: WARNING cinder.keymgr.conf_key_mgr [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] This key manager is insecure and is not recommended for production deployments: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' Sep 27 09:46:53.995778 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_db.sqlalchemy.engines [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138302) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:46:54.002090 ncdv-master-1 cinder-volume[138302]: INFO cinder.rpc [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version. Sep 27 09:46:54.007001 ncdv-master-1 cinder-volume[138302]: INFO cinder.rpc [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Automatically selected cinder-volume RPC version 3.18 as minimum service version. Sep 27 09:46:54.007712 ncdv-master-1 cinder-volume[138302]: WARNING stevedore.named [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager Sep 27 09:46:54.008127 ncdv-master-1 cinder-volume[138302]: WARNING castellan.key_manager [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' Sep 27 09:46:54.010279 ncdv-master-1 cinder-volume[138302]: INFO cinder.volume.manager [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Image-volume cache enabled for host ncdv-master-1@ceph. Sep 27 09:46:54.017642 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_db.sqlalchemy.engines [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138302) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:46:54.040691 ncdv-master-1 cinder-volume[138302]: INFO oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Starting 1 workers Sep 27 09:46:54.046632 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Started child 138473 {{(pid=138302) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}} Sep 27 09:46:54.047814 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Full set of CONF: {{(pid=138302) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}} Sep 27 09:46:54.048173 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ******************************************************************************** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 09:46:54.048711 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] Configuration options gathered from: {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 09:46:54.049258 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 09:46:54.050044 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 09:46:54.050493 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ================================================================================ {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 09:46:54.051410 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] allow_availability_zone_fallback = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.051961 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] allow_compression_on_image_upload = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.052633 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] allowed_direct_url_schemes = [] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.053213 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] api_rate_limit = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.053856 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] auth_strategy = keystone {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.054490 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] az_cache_duration = 3600 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.055036 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_availability_zone = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.055575 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_host = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.056128 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_name = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.056705 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_stats_polling_interval = 60 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.057265 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backup_api_class = cinder.backup.api.API {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.057811 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backup_manager = cinder.backup.manager.BackupManager {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.058377 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backup_name_template = backup-%s {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.058914 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backup_use_same_host = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.059372 ncdv-master-1 cinder-volume[138473]: INFO cinder.service [-] Starting cinder-volume node (version 21.1.0) Sep 27 09:46:54.060023 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backup_use_temp_snapshot = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.060580 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] chap_password = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.061161 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] chap_username = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.061693 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.062241 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cinder_internal_tenant_project_id = cd30607a8a404dd2a608b2ec915dc81d {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.062804 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cinder_internal_tenant_user_id = b324af05ec3e4b35a9bfc188b3b3abec {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.063330 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cloned_volume_same_az = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.063879 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cluster = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.064540 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] compression_format = gzip {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.065131 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] compute_api_class = cinder.compute.nova.API {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.065782 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] config_dir = [] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.066251 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] config_file = ['/etc/cinder/cinder.conf'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.066796 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] config_source = [] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.067322 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] consistencygroup_api_class = cinder.consistencygroup.api.API {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.067883 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] control_exchange = openstack {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.068440 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] debug = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.068966 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] default_availability_zone = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.069487 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] default_group_type = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.070394 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.070948 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] default_volume_type = ceph {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.071470 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] driver_client_cert = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.072209 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] driver_client_cert_key = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.072996 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] driver_data_namespace = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.073774 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] driver_ssl_cert_path = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.075037 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] driver_ssl_cert_verify = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.075780 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] driver_use_ssl = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.076565 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] enable_force_upload = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.077292 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] enable_new_services = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.077878 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] enable_unsupported_driver = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.078464 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] enabled_backends = ['ceph'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.079022 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] enforce_multipath_for_image_xfer = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.079381 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] filter_function = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.080144 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] glance_api_insecure = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.080525 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] glance_api_servers = ['http://10.222.0.26/image'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.080923 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] glance_api_ssl_compression = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.081274 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] glance_ca_certificates_file = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.081678 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] glance_catalog_info = image:glance:publicURL {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.082032 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] glance_certfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.082537 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] glance_core_properties = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.082939 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] glance_keyfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.083270 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] glance_num_retries = 3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.083635 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] glance_request_timeout = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.083980 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] goodness_function = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.084359 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] graceful_shutdown_timeout = 5 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.084796 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] group_api_class = cinder.group.api.API {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.085091 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] host = ncdv-master-1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.085600 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] iet_conf = /etc/iet/ietd.conf {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.086149 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] image_compress_on_upload = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.086671 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] image_conversion_address_space_limit = 1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.087226 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] image_conversion_cpu_limit = 60 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.087739 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] image_conversion_dir = /opt/stack/data/cinder/conversion {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.088252 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] image_conversion_disable = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.088804 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] image_upload_use_cinder_backend = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.089541 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] image_upload_use_internal_tenant = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.089541 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] image_volume_cache_enabled = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.089864 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] image_volume_cache_max_count = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.090242 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] image_volume_cache_max_size_gb = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.090611 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] init_host_max_objects_retrieval = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.090976 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] instance_format = [instance: %(uuid)s] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.091353 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.091731 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] iscsi_iotype = fileio {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.092116 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] iscsi_secondary_ip_addresses = [] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.092470 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] iscsi_target_flags = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.092866 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] iscsi_write_cache = on {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.093216 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] iser_helper = tgtadm {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.093973 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] iser_ip_address = 10.222.0.26 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.094539 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] iser_port = 3260 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.095070 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.095683 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] log_config_append = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.096233 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.096773 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] log_dir = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.097300 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] log_file = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.097616 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Starting volume driver RBDDriver (1.3.0) Sep 27 09:46:54.097938 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] log_options = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.098450 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] log_rotate_interval = 1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.099009 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] log_rotate_interval_type = days {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.099538 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] log_rotation_type = none {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.099830 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:46:54.100159 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.100621 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.101173 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.101692 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.102128 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.102674 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] max_age = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.103162 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] max_logfile_count = 30 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.103667 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] max_logfile_size_mb = 200 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.104192 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] max_over_subscription_ratio = 20.0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.104760 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] message_reap_interval = 86400 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.105276 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] message_ttl = 2592000 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.105808 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] migration_create_volume_timeout_secs = 300 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.106300 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] monkey_patch = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.106812 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] monkey_patch_modules = [] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.107263 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] my_ip = 10.222.0.26 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.107767 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] no_snapshot_gb_quota = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.108262 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] num_iser_scan_tries = 3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.108825 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] num_shell_tries = 3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.109330 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] num_volume_device_scan_tries = 3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.109874 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] nvmet_ns_id = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.110236 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] nvmet_port_id = 1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.110627 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] osapi_max_limit = 1000 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.111017 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] osapi_volume_ext_list = [] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.111448 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] osapi_volume_extension = ['cinder.api.contrib.standard_extensions'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.111834 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] osapi_volume_listen = 0.0.0.0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.112368 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] osapi_volume_listen_port = 8776 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.112915 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] osapi_volume_use_ssl = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.113453 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] osapi_volume_workers = 2 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.113974 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] per_volume_size_limit = -1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.114478 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] periodic_fuzzy_delay = 60 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.115004 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] periodic_interval = 60 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.115585 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] project_id_regex = [0-9a-f\-]+ {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.116068 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] public_endpoint = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.116554 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] publish_errors = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.117078 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] quota_backup_gigabytes = 1000 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.117603 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] quota_backups = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.118114 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] quota_consistencygroups = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.118650 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] quota_driver = cinder.quota.DbQuotaDriver {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.119166 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] quota_gigabytes = 1000 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.119666 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] quota_groups = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.120194 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] quota_snapshots = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.120575 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] quota_volumes = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.120966 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] rate_limit_burst = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.121350 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] rate_limit_except_level = CRITICAL {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.121702 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] rate_limit_interval = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.122050 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] reinit_driver_count = 3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.122418 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] replication_device = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.122768 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] report_discard_supported = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.123111 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] report_interval = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.123567 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] reservation_clean_interval = 86400 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.123862 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] reservation_expire = 86400 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.124216 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] reserved_percentage = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.124578 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] resource_query_filters_file = /etc/cinder/resource_filters.json {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.124955 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] rootwrap_config = /etc/cinder/rootwrap.conf {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.125358 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] rpc_response_timeout = 120 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.125723 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] run_external_periodic_tasks = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.126084 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] scheduler_manager = cinder.scheduler.manager.SchedulerManager {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.126440 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] scst_target_driver = iscsi {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.126909 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] scst_target_iqn_name = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.127202 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] service_down_time = 60 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.127560 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] snapshot_name_template = snapshot-%s {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.128107 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] snapshot_same_host = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.128461 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] split_loggers = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.128779 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] state_path = /opt/stack/data/cinder {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.129138 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] storage_availability_zone = nova {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.129500 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] storage_protocol = iSCSI {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.129886 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] syslog_log_facility = LOG_USER {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.130248 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] target_helper = lioadm {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.130699 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] target_ip_address = 10.222.0.26 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.131235 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] target_port = 3260 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.131784 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] target_prefix = iqn.2010-10.org.openstack: {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.132159 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] target_protocol = iscsi {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.132784 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] trace_flags = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.133278 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] transfer_api_class = cinder.transfer.api.API {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.133898 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] transport_url = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.134253 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] until_refresh = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.134708 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] use_chap_auth = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.135160 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] use_default_quota_class = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.135463 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] use_eventlog = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.136032 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] use_journal = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.136384 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] use_json = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.136801 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] use_multipath_for_image_xfer = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.137204 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] use_stderr = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.137572 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] use_syslog = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.137994 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] verify_glance_signatures = enabled {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.138353 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_api_class = cinder.volume.api.API {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.138697 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_backend_name = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.139077 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_clear = zero {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.139424 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_clear_ionice = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.139763 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_clear_size = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.140123 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.140469 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_copy_bps_limit = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.140860 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_dd_blocksize = 1M {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.141227 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_manager = cinder.volume.manager.VolumeManager {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.141577 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_name_template = volume-%s {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.141970 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_service_inithost_offload = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.142369 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volume_usage_audit_period = month {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.142952 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] volumes_dir = /opt/stack/data/cinder/volumes {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.143323 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] watch_log_file = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.143656 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] zoning_mode = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:46:54.144358 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] os_brick.lock_path = /opt/stack/data/cinder {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.145076 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.145705 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.146271 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_concurrency.disable_process_locking = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.146803 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_concurrency.lock_path = /opt/stack/data/cinder {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.147288 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.147717 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.148047 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.148413 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.148797 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.149229 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.admin_password = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.149608 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.149963 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.admin_token = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.150308 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.admin_user = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.150691 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.auth_admin_prefix = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.151051 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.151422 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.auth_port = 35357 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.151811 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.auth_protocol = https {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.152217 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.auth_section = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.152587 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.auth_type = password {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.152983 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.auth_uri = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.153349 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.auth_version = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.153709 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.cache = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.154095 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.154441 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.certfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.154800 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.delay_auth_decision = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.155175 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.155522 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.http_connect_timeout = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.155870 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.156236 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.identity_uri = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.156584 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.include_service_catalog = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.157208 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.insecure = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.157579 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.interface = public {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.157989 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.keyfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.158363 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.158716 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.159230 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.159584 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.159942 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.160407 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.160704 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.161310 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.161867 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.162384 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.region_name = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.162907 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.163409 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.service_token_roles_required = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.163906 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.service_type = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.164420 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.token_cache_time = 300 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.164964 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.165575 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.backend = dogpile.cache.null {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.166463 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.backend_argument = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.166998 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.config_prefix = cache.oslo {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.168350 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.dead_timeout = 60.0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.168910 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.debug_cache_backend = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.169471 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.enable_retry_client = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.169832 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.enable_socket_keepalive = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.170186 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.enabled = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.170573 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.expiration_time = 600 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.170925 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.hashclient_retry_attempts = 2 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.171311 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.hashclient_retry_delay = 1.0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.171677 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.memcache_dead_retry = 300 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.172057 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.memcache_password = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.172424 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.172820 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.173171 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.memcache_pool_maxsize = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.173651 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.174086 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.memcache_sasl_enabled = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.174460 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.memcache_servers = ['localhost:11211'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.174819 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.memcache_socket_timeout = 1.0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.175559 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.memcache_username = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.175945 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.proxies = [] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.176299 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.retry_attempts = 2 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.176708 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.retry_delay = 0.0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.177056 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.socket_keepalive_count = 1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.177411 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.socket_keepalive_idle = 1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.177761 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.socket_keepalive_interval = 1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.178114 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.tls_allowed_ciphers = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.178459 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.tls_cafile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.178810 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.tls_certfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.179166 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.tls_enabled = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.179531 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cache.tls_keyfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.179927 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] coordination.backend_url = etcd3+http://10.222.0.26:2379 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.180374 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.backend = sqlalchemy {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.180696 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.connection = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.181267 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.connection_debug = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.181806 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.connection_parameters = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.182334 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.connection_recycle_time = 3600 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.182846 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.connection_trace = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.183346 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:46:54.183465 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.db_inc_retry_interval = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.183937 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.db_max_retries = 20 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.184439 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.db_max_retry_interval = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.184976 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.db_retry_interval = 1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.185498 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.max_overflow = 50 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.186011 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.max_pool_size = 5 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.186525 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.max_retries = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.187018 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.mysql_enable_ndb = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.187547 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.188053 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.pool_timeout = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.188573 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.retry_interval = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.189152 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.slave_connection = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.189670 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.sqlite_synchronous = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.190175 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] database.use_db_reconnect = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.190731 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_policy.enforce_new_defaults = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.191242 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_policy.enforce_scope = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.191784 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_policy.policy_default_rule = default {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.192311 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.192888 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_policy.policy_file = policy.yaml {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.193470 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.193971 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.194338 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.194716 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.195093 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.195531 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] profiler.connection_string = messaging:// {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.195825 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] profiler.enabled = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.196190 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] profiler.es_doc_type = notification {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.196544 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] profiler.es_scroll_size = 10000 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.196931 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] profiler.es_scroll_time = 2m {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.197292 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] profiler.filter_error_trace = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.197651 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] profiler.hmac_keys = SECRET_KEY {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.198011 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] profiler.sentinel_service_name = mymaster {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.198664 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] profiler.socket_timeout = 0.1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.198922 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] profiler.trace_sqlalchemy = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.199332 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] privsep_osbrick.capabilities = [21] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.199707 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] privsep_osbrick.group = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.200063 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] privsep_osbrick.helper_command = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.200416 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] privsep_osbrick.logger_name = oslo_privsep.daemon {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.200797 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.201160 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] privsep_osbrick.user = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.201559 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cinder_sys_admin.capabilities = [0, 1, 2, 3, 12, 21] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.202067 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cinder_sys_admin.group = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.202573 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cinder_sys_admin.helper_command = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.203082 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cinder_sys_admin.logger_name = oslo_privsep.daemon {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.203587 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.204101 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] cinder_sys_admin.user = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.204705 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] service_user.auth_section = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.205074 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] service_user.auth_type = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.205423 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] service_user.cafile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.205773 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] service_user.certfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.206139 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] service_user.collect_timing = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.206494 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] service_user.insecure = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.206840 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] service_user.keyfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.207190 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] service_user.send_service_user_token = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.207637 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] service_user.split_loggers = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.207896 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] service_user.timeout = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.208345 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.backend_availability_zone = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.208770 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.backend_native_threads_pool_size = 20 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.209151 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.backup_use_temp_snapshot = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.209503 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.chap_password = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.209870 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.chap_username = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.210249 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.210617 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.210984 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.211332 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.driver_client_cert = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.211677 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.driver_client_cert_key = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.212031 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.driver_data_namespace = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.212568 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.213168 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.213663 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.driver_use_ssl = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.214156 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.enable_deferred_deletion = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.214647 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.enable_unsupported_driver = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.215153 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.215658 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.extra_capabilities = {} {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.216158 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.filter_function = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.216642 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.goodness_function = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.217256 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.iet_conf = /etc/iet/ietd.conf {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.217788 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.218292 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.218896 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.219699 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.220718 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.221301 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.iscsi_iotype = fileio {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.221677 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.222048 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.iscsi_target_flags = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.222420 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.iscsi_write_cache = on {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.222784 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.iser_helper = tgtadm {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.223332 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.iser_ip_address = 10.222.0.26 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.223690 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.iser_port = 3260 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.224059 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.224425 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.224804 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.225354 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.num_shell_tries = 3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.225711 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.226063 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.nvmet_ns_id = 10 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.226430 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.nvmet_port_id = 1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.226765 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.227111 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rados_connection_interval = 5 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.227475 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rados_connection_retries = 3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.227944 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rbd_ceph_conf = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.228225 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.228601 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.228969 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.229325 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.232768 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rbd_pool = rbd {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.232768 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rbd_secret_uuid = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.232768 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.232768 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.rbd_user = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.232768 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.232768 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.replication_device = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.233362 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.report_discard_supported = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.233362 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.233362 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.reserved_percentage = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.233362 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.scst_target_driver = iscsi {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.233362 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.scst_target_iqn_name = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.233718 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.storage_protocol = iSCSI {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.234000 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.suppress_requests_ssl_warnings = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.234364 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.target_helper = tgtadm {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.234887 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.target_ip_address = 10.222.0.26 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.235319 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.target_port = 3260 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.235734 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.236179 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.target_protocol = iscsi {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.236711 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.trace_flags = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.237239 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.unique_fqdn_network = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.237803 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.use_chap_auth = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.238337 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.238708 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.volume_backend_name = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.239102 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.volume_clear = zero {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.239459 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.volume_clear_ionice = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.239899 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.volume_clear_size = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244384 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244384 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244384 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244384 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244384 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] backend_defaults.volumes_dir = /opt/stack/data/cinder/volumes {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244384 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_reports.file_event_handler = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244896 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244896 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_reports.log_dir = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244896 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.backend_availability_zone = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244896 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.backend_host = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244896 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.backend_native_threads_pool_size = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244896 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.backup_use_temp_snapshot = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.244896 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.chap_password = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.245379 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.chap_username = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.245379 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.chiscsi_conf = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.245639 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.deferred_deletion_delay = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.246258 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.deferred_deletion_purge_interval = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.246768 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.driver_client_cert = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.247291 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.driver_client_cert_key = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.247625 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.driver_data_namespace = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.247995 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.driver_ssl_cert_path = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.248445 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.driver_ssl_cert_verify = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.248900 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.driver_use_ssl = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.249270 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.enable_deferred_deletion = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.249633 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.enable_unsupported_driver = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.250006 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.enforce_multipath_for_image_xfer = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.250339 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.extra_capabilities = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.250740 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.filter_function = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.251090 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.goodness_function = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.251455 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.iet_conf = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.252391 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.image_upload_use_cinder_backend = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.252795 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.image_upload_use_internal_tenant = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.253406 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.image_volume_cache_enabled = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.253911 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.image_volume_cache_max_count = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.254391 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.image_volume_cache_max_size_gb = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.254871 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.iscsi_iotype = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.255359 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.iscsi_secondary_ip_addresses = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.255883 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.iscsi_target_flags = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.256410 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.iscsi_write_cache = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.256960 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.iser_helper = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.257439 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.iser_ip_address = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.257932 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.iser_port = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.258629 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.iser_target_prefix = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.259153 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.max_over_subscription_ratio = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.259502 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.num_iser_scan_tries = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.260001 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.num_shell_tries = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.260359 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.num_volume_device_scan_tries = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.260731 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.nvmet_ns_id = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.261081 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.nvmet_port_id = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.261455 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rados_connect_timeout = -1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.261797 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rados_connection_interval = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.262143 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rados_connection_retries = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.262511 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rbd_ceph_conf = /etc/ceph/ceph.conf {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.262855 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rbd_cluster_name = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.263200 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rbd_exclusive_cinder_pool = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.263565 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rbd_flatten_volume_from_snapshot = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.263924 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rbd_max_clone_depth = 5 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.264283 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rbd_pool = volumes {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.264640 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rbd_secret_uuid = a0a9ff1d-f350-4984-ac7b-a927bddf8534 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.265028 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rbd_store_chunk_size = 4 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.265384 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.rbd_user = cinder {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.265736 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.replication_connect_timeout = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.266080 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.replication_device = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.266423 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.report_discard_supported = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.266765 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.report_dynamic_total_capacity = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.267106 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.reserved_percentage = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.267501 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.scst_target_driver = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.268118 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.scst_target_iqn_name = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.268485 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.storage_protocol = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.268873 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.suppress_requests_ssl_warnings = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.269213 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.target_helper = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.269559 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.target_ip_address = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.269934 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.target_port = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.270266 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.target_prefix = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.270608 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.target_protocol = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.270945 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.trace_flags = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.271285 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.unique_fqdn_network = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.271629 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.use_chap_auth = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.271969 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.use_multipath_for_image_xfer = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.272337 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.volume_backend_name = ceph {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.272696 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.volume_clear = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.273052 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.volume_clear_ionice = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.273396 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.volume_clear_size = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.273736 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.volume_copy_blkio_cgroup_name = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.274077 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.volume_copy_bps_limit = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.274461 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.volume_dd_blocksize = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.274791 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.volume_driver = cinder.volume.drivers.rbd.RBDDriver {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.275133 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ceph.volumes_dir = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.275572 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.275945 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.276311 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.276683 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.277055 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.277409 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.277790 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.278133 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.278477 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.278850 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.279237 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.279590 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.279976 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.280331 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.280777 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.281155 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.281530 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.281847 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.282198 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.282545 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.282923 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.283275 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.283618 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.283980 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.284331 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.284729 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.ssl = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.285112 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.285447 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.285813 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.286175 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.286536 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_rabbit.ssl_version = {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.286859 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_notifications.driver = [] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.287234 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_notifications.retry = -1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.287634 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.287977 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] oslo_messaging_notifications.transport_url = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.288310 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] key_manager.backend = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.288605 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] key_manager.fixed_key = **** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.289088 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.289509 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.barbican_api_version = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.289798 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.barbican_endpoint = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.290246 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.barbican_endpoint_type = public {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.290635 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.barbican_region_name = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.290843 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.cafile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.291527 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.certfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.291888 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.collect_timing = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.292232 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.insecure = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.292568 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.keyfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.293689 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.number_of_retries = 60 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.294059 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.retry_delay = 1 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.294428 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.send_service_user_token = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.294779 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.split_loggers = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.295165 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.timeout = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.295772 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.verify_ssl = True {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.296127 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican.verify_ssl_path = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.296510 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican_service_user.auth_section = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.296883 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican_service_user.auth_type = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.297222 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican_service_user.cafile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.297585 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican_service_user.certfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.297933 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican_service_user.collect_timing = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.298283 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican_service_user.insecure = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.298673 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican_service_user.keyfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.299071 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican_service_user.split_loggers = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.299374 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] barbican_service_user.timeout = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.299752 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.approle_role_id = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.300091 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.approle_secret_id = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.300436 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.cafile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.300812 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.certfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.301164 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.collect_timing = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.301506 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.insecure = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.301845 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.keyfile = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.302211 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.kv_mountpoint = secret {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.302614 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.kv_version = 2 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.302941 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.namespace = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.303285 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.root_token_id = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.303836 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.split_loggers = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.304186 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.ssl_ca_crt_file = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.304528 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.timeout = None {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.304923 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.use_ssl = False {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.305285 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:46:54.305570 ncdv-master-1 cinder-volume[138302]: DEBUG oslo_service.service [None req-4219cfe5-7e04-48d0-8572-088ccf90d880 None None] ******************************************************************************** {{(pid=138302) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 09:46:54.309263 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:46:54.337364 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.driver [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Driver hasn't implemented _init_vendor_properties() Sep 27 09:46:54.337646 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.driver [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Initialized capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.3.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:e8b2ca7b-8933-4391-a7b9-e76577b88317:cinder:volumes', 'backend_state': 'up', 'qos_support': True, 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=138473) init_capabilities /opt/stack/cinder/cinder/volume/driver.py:812}} Sep 27 09:46:54.352439 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_db.sqlalchemy.engines [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138473) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:46:54.355025 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writer thread running {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:46:54.388959 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.db.sqlalchemy.api [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Building query based on filter {{(pid=138473) _process_snaps_filters /opt/stack/cinder/cinder/db/sqlalchemy/api.py:3885}} Sep 27 09:46:54.413153 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.image_utils [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Configuration image_conversion_dir is None or the path doesn't exist. {{(pid=138473) cleanup_temporary_file /opt/stack/cinder/cinder/image/image_utils.py:991}} Sep 27 09:46:54.419847 ncdv-master-1 cinder-volume[138473]: INFO cinder.keymgr.migration [None req-492153a8-3685-4446-afd3-bfc645e4d04a None None] Not migrating encryption keys because the ConfKeyManager is still in use. Sep 27 09:46:54.427914 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:46:54.459721 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:46:54.488786 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:46:54.514474 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Driver initialization completed successfully. Sep 27 09:46:54.515722 ncdv-master-1 cinder-volume[138473]: INFO cinder.manager [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Initiating service 3 cleanup Sep 27 09:46:54.526778 ncdv-master-1 cinder-volume[138473]: INFO cinder.manager [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Service 3 cleanup completed. Sep 27 09:46:54.527529 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.service [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Creating RPC server for service cinder-volume {{(pid=138473) start /opt/stack/cinder/cinder/service.py:238}} Sep 27 09:46:54.537388 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.service [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=138473) start /opt/stack/cinder/cinder/service.py:244}} Sep 27 09:46:54.583346 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Initializing RPC dependent components of volume driver RBDDriver (1.3.0) Sep 27 09:46:54.584650 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:46:54.634732 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:46:54.690925 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-296dd995-a41a-432b-9a06-64cfe29ed431 None None] Driver post RPC initialization completed successfully. Sep 27 09:47:04.032524 ncdv-master-1 cinder-volume[138302]: DEBUG dbcounter [-] [138302] Writing DB stats cinder:SELECT=16,cinder:INSERT=1 {{(pid=138302) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:04.683776 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=23,cinder:INSERT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:04.686733 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:34.710366 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:44.714924 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:14.716599 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:14.727384 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:29.695279 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:48:29.696220 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:48:29.720919 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:48:29.746860 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:48:44.717478 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:48:54.718508 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:14.711699 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:24.723273 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:29.691728 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:49:29.692455 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:49:29.718683 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:49:29.743270 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:49:44.724988 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:54.732778 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:50:24.716414 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:50:29.688911 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:50:29.690135 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:50:29.725482 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:50:29.752514 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:50:34.723804 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:51:04.714827 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:51:14.712798 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:51:29.687932 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:51:29.689134 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:51:29.713034 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:51:29.737746 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:51:34.717080 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:51:54.699191 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:52:24.702218 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:52:29.683933 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:52:29.684753 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:52:29.715656 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:52:29.744322 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:52:34.698505 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:52:54.702489 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:53:14.705946 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:53:29.680243 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:53:29.681021 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:53:29.711666 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:53:29.739108 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:53:34.709067 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:53:44.717084 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:54:04.709258 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:54:14.727481 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:54:29.684963 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:54:29.685932 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:54:29.714261 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:54:29.740175 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:54:34.718761 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:54:44.726891 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:55:14.725378 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:55:24.725175 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:55:29.686277 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:55:29.687008 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:55:29.727501 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:55:29.765856 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:55:44.736090 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:55:44.738400 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:14.743617 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:24.739354 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:29.682171 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:56:29.682931 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:56:29.764483 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:56:29.831750 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:56:34.747273 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:44.758806 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:04.760578 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:29.681675 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:57:29.682403 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:57:29.748131 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:57:29.819174 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:57:34.754762 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:44.758037 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:14.748796 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:24.743998 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:29.686706 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:58:29.688075 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:58:29.819957 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:58:29.901856 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:58:44.742770 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:54.734942 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:04.746499 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:24.743226 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:29.687056 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 09:59:29.687781 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:59:29.769158 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 09:59:29.856279 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 09:59:34.760120 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=9,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:54.747619 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:04.753317 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:14.772292 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:29.683343 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:00:29.685113 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:00:29.762818 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:00:29.842313 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:00:34.773388 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:04.764857 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:14.758308 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:24.786232 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:29.681371 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:01:29.682419 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:01:29.761662 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:01:29.845869 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:01:44.765442 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:54.776286 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:14.778010 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:29.687502 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:02:29.688222 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:02:29.758448 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:02:29.836559 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:02:33.412555 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:02:33.519270 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Flow 'volume_create_manager' (28eac0a2-dc36-491d-8463-2fe3c366f83f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:02:33.535777 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (94321cdd-72c7-45a4-94f6-5ad8a02c687a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:33.627007 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:33.693802 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (94321cdd-72c7-45a4-94f6-5ad8a02c687a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:02:33Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-2013202373',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ff80fef3-6188-4603-afd2-55aa0b605e0c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='183cb066566c4510945d6eb0d41a2853',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:02:33Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:02:33Z,use_quota=True,user_id='06dc3376ada04ffe922c9023f4182259',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:33.703715 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cd23a56a-411a-4baf-bf72-9510a887a687) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:33.722314 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cd23a56a-411a-4baf-bf72-9510a887a687) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:33.733167 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7a58f28a-37c1-4d46-a28e-bfb0b9d72734) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:33.913816 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7a58f28a-37c1-4d46-a28e-bfb0b9d72734) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'ff80fef3-6188-4603-afd2-55aa0b605e0c', 'volume_name': 'volume-ff80fef3-6188-4603-afd2-55aa0b605e0c', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:33.935832 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f2540a37-eb73-415d-a276-6ab0e43b5a67) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:33.947690 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f2540a37-eb73-415d-a276-6ab0e43b5a67) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:33.961196 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (209f5e58-e2c8-4c9a-9748-f6d3cb62c279) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:33.963282 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Volume ff80fef3-6188-4603-afd2-55aa0b605e0c: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-ff80fef3-6188-4603-afd2-55aa0b605e0c', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:02:33.965139 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Cloning ff80fef3-6188-4603-afd2-55aa0b605e0c from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:02:33.969107 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:02:33.995247 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.028s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:02:34.015605 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Preparing cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:891}} Sep 27 10:02:34.016754 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:02:49.601404 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:02:49.623495 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Flow 'volume_create_manager' (f70d2c17-6538-470a-af96-53212fe0b7c9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:02:49.630022 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (494fa6c0-afed-426a-8cd4-f3a4f1418aeb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:49.648010 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (494fa6c0-afed-426a-8cd4-f3a4f1418aeb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:02:49Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachSCSIVolumeTestJSON-volume-644106088',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=814c3373-89e6-4c1a-b1e0-8ddb0523d638,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5de10e92e2024a24880c7205697e5732',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:02:50Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:02:50Z,use_quota=True,user_id='0d563feacff84b4c9927ec759fb365be',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:49.653776 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d5777d1d-20ef-4a59-9080-b9cdc771b7c6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:49.657954 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d5777d1d-20ef-4a59-9080-b9cdc771b7c6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:49.663200 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6b3901f6-5746-4e49-b982-ac94c233591e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:49.667236 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6b3901f6-5746-4e49-b982-ac94c233591e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '814c3373-89e6-4c1a-b1e0-8ddb0523d638', 'volume_name': 'volume-814c3373-89e6-4c1a-b1e0-8ddb0523d638', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:49.671901 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (59f57e9d-879d-41e0-9346-c111fe9d5f5a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:49.675782 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (59f57e9d-879d-41e0-9346-c111fe9d5f5a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:49.680583 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (618e6422-acf2-4904-9a7c-3056ffc37872) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:49.681700 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Volume 814c3373-89e6-4c1a-b1e0-8ddb0523d638: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-814c3373-89e6-4c1a-b1e0-8ddb0523d638', 'volume_size': 1} Sep 27 10:02:49.685247 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] creating volume 'volume-814c3373-89e6-4c1a-b1e0-8ddb0523d638' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:02:49.685247 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:02:49.918725 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:02:49.919103 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:02:49.923895 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (618e6422-acf2-4904-9a7c-3056ffc37872) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-814c3373-89e6-4c1a-b1e0-8ddb0523d638', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:49.940897 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (31506df7-fe85-430a-927c-b28f806e0aa9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:49.967430 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Volume volume-814c3373-89e6-4c1a-b1e0-8ddb0523d638 (814c3373-89e6-4c1a-b1e0-8ddb0523d638): created successfully Sep 27 10:02:49.975980 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (31506df7-fe85-430a-927c-b28f806e0aa9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:49.986597 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Flow 'volume_create_manager' (f70d2c17-6538-470a-af96-53212fe0b7c9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:02:50.017159 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-463017f2-5b72-4447-924d-867a2659cc4c tempest-AttachSCSIVolumeTestJSON-884590422 None] Created volume successfully. Sep 27 10:02:51.217708 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.image_utils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Image fetch details: dest /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph, size 616.00 MB, duration 17.19 sec {{(pid=138473) fetch /opt/stack/cinder/cinder/image/image_utils.py:587}} Sep 27 10:02:51.218839 ncdv-master-1 cinder-volume[138473]: INFO cinder.image.image_utils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Image download 616.00 MB at 35.84 MB/s Sep 27 10:02:51.256992 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:02:51.435127 ncdv-master-1 sudo[153515]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph Sep 27 10:02:51.436129 ncdv-master-1 sudo[153515]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 27 10:02:53.111710 ncdv-master-1 sudo[153515]: pam_unix(sudo:session): session closed for user root Sep 27 10:02:53.114470 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph" returned: 0 in 1.857s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:02:53.115955 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.image_utils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Temporary image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 is fetched for user 06dc3376ada04ffe922c9023f4182259. {{(pid=138473) fetch /opt/stack/cinder/cinder/image/image_utils.py:1124}} Sep 27 10:02:53.201748 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:02:53.362883 ncdv-master-1 sudo[153556]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph Sep 27 10:02:53.363768 ncdv-master-1 sudo[153556]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 27 10:02:53.504459 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c820eb30-9f33-435d-80f6-ef5202282a0d req-af924cb5-a95d-4a3c-82b3-6ecc93cef615 tempest-AttachSCSIVolumeTestJSON-884590422 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:02:54.950614 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c820eb30-9f33-435d-80f6-ef5202282a0d req-af924cb5-a95d-4a3c-82b3-6ecc93cef615 tempest-AttachSCSIVolumeTestJSON-884590422 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.446s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:02:54.951530 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-c820eb30-9f33-435d-80f6-ef5202282a0d req-af924cb5-a95d-4a3c-82b3-6ecc93cef615 tempest-AttachSCSIVolumeTestJSON-884590422 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-814c3373-89e6-4c1a-b1e0-8ddb0523d638', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '814c3373-89e6-4c1a-b1e0-8ddb0523d638', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:02:55.017657 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=21,cinder:UPDATE=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:55.038611 ncdv-master-1 sudo[153556]: pam_unix(sudo:session): session closed for user root Sep 27 10:02:55.043404 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-c820eb30-9f33-435d-80f6-ef5202282a0d req-af924cb5-a95d-4a3c-82b3-6ecc93cef615 tempest-AttachSCSIVolumeTestJSON-884590422 None] Connection info returned from driver {'name': 'volumes/volume-814c3373-89e6-4c1a-b1e0-8ddb0523d638', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '814c3373-89e6-4c1a-b1e0-8ddb0523d638', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '0c315aea-fe66-45a1-80f6-8e860f524075'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:02:55.119204 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph" returned: 0 in 1.917s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:02:55.120255 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] creating volume 'volume-ff80fef3-6188-4603-afd2-55aa0b605e0c' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:02:55.121294 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:02:55.215293 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-c820eb30-9f33-435d-80f6-ef5202282a0d req-af924cb5-a95d-4a3c-82b3-6ecc93cef615 tempest-AttachSCSIVolumeTestJSON-884590422 None] attachment_update completed successfully. Sep 27 10:02:55.245151 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:02:55.245762 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:02:55.271459 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Attempting download of c1623d3a-c1bb-4649-9910-5deadb8a1ff6 ((None, None)) to volume ff80fef3-6188-4603-afd2-55aa0b605e0c. {{(pid=138473) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1190}} Sep 27 10:02:55.321720 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_download_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_mkwtq2jp {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:02:55.472234 ncdv-master-1 sudo[153601]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_download_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_mkwtq2jp Sep 27 10:02:55.473201 ncdv-master-1 sudo[153601]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 27 10:02:56.844545 ncdv-master-1 sudo[153601]: pam_unix(sudo:session): session closed for user root Sep 27 10:02:56.846836 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_download_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_mkwtq2jp" returned: 0 in 1.525s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:02:56.848778 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:02:56.933623 ncdv-master-1 sudo[153615]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph Sep 27 10:02:56.934797 ncdv-master-1 sudo[153615]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 27 10:02:58.198543 ncdv-master-1 sudo[153615]: pam_unix(sudo:session): session closed for user root Sep 27 10:02:58.201452 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph" returned: 0 in 1.353s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:02:58.202213 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.image_utils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] c1623d3a-c1bb-4649-9910-5deadb8a1ff6 was vhdx, converting to raw {{(pid=138473) fetch_to_volume_format /opt/stack/cinder/cinder/image/image_utils.py:825}} Sep 27 10:02:58.203492 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph /opt/stack/data/cinder/conversion/tmp_f7x8rt9 {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:02:58.222935 ncdv-master-1 sudo[153621]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph /opt/stack/data/cinder/conversion/tmp_f7x8rt9 Sep 27 10:02:58.223733 ncdv-master-1 sudo[153621]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 27 10:02:59.770940 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:02:59.818259 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Flow 'volume_create_manager' (52e354c7-5867-4552-a764-15a7ea4f2f7d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:02:59.825423 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5175cca7-089b-4d8b-b05c-61dc8f743fa8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:59.845180 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5175cca7-089b-4d8b-b05c-61dc8f743fa8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:02:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveMigrationTest-volume-1325605419',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ec0980b5824b425fbad9b8554cf5c42c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:03:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:03:00Z,use_quota=True,user_id='dce01d3212934161a38be853b4cd93ae',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:59.851391 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a1134249-18ce-4759-bf89-b31235011b0b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:59.856421 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a1134249-18ce-4759-bf89-b31235011b0b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:59.862718 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (207c05eb-4cbb-4c6c-a7f7-16eb13223cfc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:59.869250 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (207c05eb-4cbb-4c6c-a7f7-16eb13223cfc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'volume_name': 'volume-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:59.874768 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b1a53bbd-219d-4847-8dca-971ced0aa096) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:59.879853 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b1a53bbd-219d-4847-8dca-971ced0aa096) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:02:59.885307 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (98a736f2-46b4-4245-a20b-27f542981ffd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:02:59.886786 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Volume ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'volume_size': 1} Sep 27 10:02:59.887374 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] creating volume 'volume-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:02:59.888469 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:03:00.042029 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:03:00.042415 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:03:00.046503 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (98a736f2-46b4-4245-a20b-27f542981ffd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:00.059225 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2875a7a2-0462-4e75-af76-2bf46f1273c4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:00.093170 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Volume volume-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7 (ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7): created successfully Sep 27 10:03:00.097589 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2875a7a2-0462-4e75-af76-2bf46f1273c4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:00.107072 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Flow 'volume_create_manager' (52e354c7-5867-4552-a764-15a7ea4f2f7d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:03:00.136073 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d58eccb1-5fbb-4d11-b2ec-ae96891bc8cd tempest-LiveMigrationTest-1517090534 None] Created volume successfully. Sep 27 10:03:00.496797 ncdv-master-1 sudo[153621]: pam_unix(sudo:session): session closed for user root Sep 27 10:03:00.498819 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph /opt/stack/data/cinder/conversion/tmp_f7x8rt9" returned: 0 in 2.295s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:03:00.499802 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:03:00.597064 ncdv-master-1 sudo[153646]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph Sep 27 10:03:00.597853 ncdv-master-1 sudo[153646]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 27 10:03:01.816741 ncdv-master-1 sudo[153646]: pam_unix(sudo:session): session closed for user root Sep 27 10:03:01.823476 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph" returned: 0 in 1.319s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:03:01.823476 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.image_utils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Image conversion details: src /opt/stack/data/cinder/conversion/image_fetch_c1623d3a-c1bb-4649-9910-5deadb8a1ff6_5py7tk0encdv-master-1@ceph, size 700.00 MB, duration 2.30 sec, destination /opt/stack/data/cinder/conversion/tmp_f7x8rt9 {{(pid=138473) _convert_image /opt/stack/cinder/cinder/image/image_utils.py:419}} Sep 27 10:03:01.823476 ncdv-master-1 cinder-volume[138473]: INFO cinder.image.image_utils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Converted 700.00 MB image at 304.84 MB/s Sep 27 10:03:01.823476 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:03:01.931473 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:03:01.934316 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:03:01.963499 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] deleting rbd volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:03:02.262076 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Running cmd (subprocess): rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmp_f7x8rt9 volume-ff80fef3-6188-4603-afd2-55aa0b605e0c --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:03:02.829764 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-a24345b5-fbe7-444a-af66-459933243e47 req-4643c785-deae-46df-a443-9040b622e978 tempest-LiveMigrationTest-1517090534 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:03:03.734058 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-a24345b5-fbe7-444a-af66-459933243e47 req-4643c785-deae-46df-a443-9040b622e978 tempest-LiveMigrationTest-1517090534 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.904s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:03:03.737424 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-a24345b5-fbe7-444a-af66-459933243e47 req-4643c785-deae-46df-a443-9040b622e978 tempest-LiveMigrationTest-1517090534 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:03:03.780623 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-a24345b5-fbe7-444a-af66-459933243e47 req-4643c785-deae-46df-a443-9040b622e978 tempest-LiveMigrationTest-1517090534 None] Connection info returned from driver {'name': 'volumes/volume-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '90f296cc-7fc0-42b9-a47b-ea9dcd3bf47b'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:03:03.784852 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=34,cinder:UPDATE=5,cinder:DELETE=2,cinder:INSERT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:03.852265 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-a24345b5-fbe7-444a-af66-459933243e47 req-4643c785-deae-46df-a443-9040b622e978 tempest-LiveMigrationTest-1517090534 None] attachment_update completed successfully. Sep 27 10:03:07.061564 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] CMD "rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmp_f7x8rt9 volume-ff80fef3-6188-4603-afd2-55aa0b605e0c --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 4.796s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:03:07.302590 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:03:07.494472 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:03:07.495000 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:03:07.495571 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Downloaded image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 ((None, None)) to volume ff80fef3-6188-4603-afd2-55aa0b605e0c successfully. {{(pid=138473) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1232}} Sep 27 10:03:07.660478 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.image_utils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Temporary image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 for user 06dc3376ada04ffe922c9023f4182259 is deleted. {{(pid=138473) fetch /opt/stack/cinder/cinder/image/image_utils.py:1128}} Sep 27 10:03:07.692850 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-556562d6-3b59-4ba2-a673-f07543d4b96b req-a19ff1fd-bf83-49fe-9e89-90a85971484f tempest-AttachSCSIVolumeTestJSON-884590422 None] Terminate volume connection completed successfully. Sep 27 10:03:07.693362 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-556562d6-3b59-4ba2-a673-f07543d4b96b req-a19ff1fd-bf83-49fe-9e89-90a85971484f tempest-AttachSCSIVolumeTestJSON-884590422 None] Deleting attachment 0c315aea-fe66-45a1-80f6-8e860f524075. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:03:07.739416 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Deprecated: Default quota for resource: volumes_ceph is set by the default quota flag: quota_volumes_ceph, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:03:07.740798 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Deprecated: Default quota for resource: gigabytes_ceph is set by the default quota flag: quota_gigabytes_ceph, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:03:07.892398 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Created reservations ['c1863622-b455-49b8-ac22-666053ce4329', '77001931-2da4-4b1f-8bd1-0ef1436f6213', '9716970a-99e0-41f9-8246-edc58aa2d30c', 'a48e4dc9-f020-41aa-acf1-dd84362ec6ee'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:03:07.920805 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Creating image volume entry: {'consistencygroup_id': None, 'updated_at': datetime.datetime(2022, 9, 27, 10, 2, 33, tzinfo=datetime.timezone.utc), 'bootable': False, 'replication_driver_data': None, 'display_name': 'image-c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'service_uuid': None, 'provider_id': None, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'provider_auth': None, 'scheduled_at': datetime.datetime(2022, 9, 27, 10, 2, 33, tzinfo=datetime.timezone.utc), 'replication_extended_status': None, 'volume_glance_metadata': [{'key': 'signature_verified', 'value': 'False'}], 'metadata': {}, 'volume_metadata': [], 'replication_status': None, 'provider_location': None, 'deleted_at': None, 'volume_type_id': 'e6016daf-4ebe-4600-800c-f0685eb2234c', 'deleted': False, 'size': 1, 'host': 'ncdv-master-1@ceph#ceph', 'previous_status': None, 'terminated_at': None, 'group_id': None, 'shared_targets': True, 'created_at': datetime.datetime(2022, 9, 27, 10, 2, 33, tzinfo=datetime.timezone.utc), 'multiattach': False, 'project_id': 'cd30607a8a404dd2a608b2ec915dc81d', 'launched_at': None, 'volume_admin_metadata': [], 'ec2_id': None, 'cluster_name': None, 'provider_geometry': None, 'encryption_key_id': None, 'availability_zone': 'nova', 'display_description': None, 'attach_status': 'detached', 'status': 'creating', 'source_volid': 'ff80fef3-6188-4603-afd2-55aa0b605e0c'}. {{(pid=138473) _clone_image_volume /opt/stack/cinder/cinder/volume/manager.py:1660}} Sep 27 10:03:08.068392 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Volume reschedule parameters: False retry: None {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:03:08.127302 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Flow 'volume_create_manager' (37f1def6-ec45-41d9-86ba-7da35e071fbc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:03:08.146825 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c2fd7d5c-d503-468e-a9dc-fffe7bbe2472) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:08.171723 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c2fd7d5c-d503-468e-a9dc-fffe7bbe2472) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:02:33Z,deleted=False,deleted_at=None,display_description=None,display_name='image-c1623d3a-c1bb-4649-9910-5deadb8a1ff6',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=78171a54-b27c-4c21-b4c6-bf0ecca1bcd1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='cd30607a8a404dd2a608b2ec915dc81d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:02:33Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=ff80fef3-6188-4603-afd2-55aa0b605e0c,status='creating',terminated_at=None,updated_at=2022-09-27T10:02:33Z,use_quota=True,user_id='06dc3376ada04ffe922c9023f4182259',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:08.179530 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a98c2f0c-96c2-424a-bca6-f9b3626a6d41) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:08.184386 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a98c2f0c-96c2-424a-bca6-f9b3626a6d41) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:08.191373 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4534ed06-a90f-4701-9929-ddae09c0fed7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:08.213909 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4534ed06-a90f-4701-9929-ddae09c0fed7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'volume_name': 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'volume_size': 1, 'source_volid': 'ff80fef3-6188-4603-afd2-55aa0b605e0c', 'source_volstatus': 'downloading'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:08.220719 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1ff01f4d-884d-49f1-b208-517fa4160153) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:08.225037 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1ff01f4d-884d-49f1-b208-517fa4160153) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:08.230139 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d88324ea-1f83-47d2-9b5c-99fe0539bad8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:08.231230 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'volume_size': 1, 'source_volid': 'ff80fef3-6188-4603-afd2-55aa0b605e0c', 'source_volstatus': 'downloading'} Sep 27 10:03:08.261673 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:03:08.328299 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] creating snapshot='volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:03:08.742284 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] cloning 'volume-ff80fef3-6188-4603-afd2-55aa0b605e0c@volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1.clone_snap' to 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:03:08.896343 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:03:08.897696 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:03:08.898224 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:03:08.932148 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:03:08.937728 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d88324ea-1f83-47d2-9b5c-99fe0539bad8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'volume_size': 1, 'source_volid': 'ff80fef3-6188-4603-afd2-55aa0b605e0c', 'source_volstatus': 'downloading'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:08.944084 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e9de9483-f67d-4260-bcab-cbf0a417f4b4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:08.958624 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Volume volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 (78171a54-b27c-4c21-b4c6-bf0ecca1bcd1): created successfully Sep 27 10:03:08.964179 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e9de9483-f67d-4260-bcab-cbf0a417f4b4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:08.971511 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Flow 'volume_create_manager' (37f1def6-ec45-41d9-86ba-7da35e071fbc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:03:08.988053 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Created volume successfully. Sep 27 10:03:09.052302 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Creating new image-volume cache entry for image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on ncdv-master-1@ceph#ceph {{(pid=138473) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:104}} Sep 27 10:03:09.072986 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] New image-volume cache entry created: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 3, 9, 56219)}. {{(pid=138473) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:129}} Sep 27 10:03:09.080597 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 35.086s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:03:09.081369 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Marking volume ff80fef3-6188-4603-afd2-55aa0b605e0c as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:03:09.093001 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to ff80fef3-6188-4603-afd2-55aa0b605e0c. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:03:09.093662 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Creating volume glance metadata for volume ff80fef3-6188-4603-afd2-55aa0b605e0c backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:03:09.181443 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (209f5e58-e2c8-4c9a-9748-f6d3cb62c279) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ff80fef3-6188-4603-afd2-55aa0b605e0c', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:09.186245 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (59cdab6b-b81a-4517-8c3d-b4beb0615641) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:09.204423 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c (ff80fef3-6188-4603-afd2-55aa0b605e0c): created successfully Sep 27 10:03:09.208265 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (59cdab6b-b81a-4517-8c3d-b4beb0615641) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:09.215019 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Flow 'volume_create_manager' (28eac0a2-dc36-491d-8463-2fe3c366f83f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:03:09.229536 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-3327d412-b8fe-40e0-a41c-f92c9ef3fd0f tempest-ServersTestBootFromVolume-1362999175 None] Created volume successfully. Sep 27 10:03:12.586020 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-3d0fc42a-c477-4d73-a429-438158b15d86 req-15f68470-81f6-48e2-8052-5c8cdacc15cd tempest-ServersTestBootFromVolume-1362999175 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:03:13.625855 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-3d0fc42a-c477-4d73-a429-438158b15d86 req-15f68470-81f6-48e2-8052-5c8cdacc15cd tempest-ServersTestBootFromVolume-1362999175 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.027s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:03:13.628027 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-3d0fc42a-c477-4d73-a429-438158b15d86 req-15f68470-81f6-48e2-8052-5c8cdacc15cd tempest-ServersTestBootFromVolume-1362999175 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-ff80fef3-6188-4603-afd2-55aa0b605e0c', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'ff80fef3-6188-4603-afd2-55aa0b605e0c', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:03:13.665871 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-3d0fc42a-c477-4d73-a429-438158b15d86 req-15f68470-81f6-48e2-8052-5c8cdacc15cd tempest-ServersTestBootFromVolume-1362999175 None] Connection info returned from driver {'name': 'volumes/volume-ff80fef3-6188-4603-afd2-55aa0b605e0c', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'ff80fef3-6188-4603-afd2-55aa0b605e0c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '3c2bfdba-ff96-41d9-8ac6-e242764807e4'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:03:13.711184 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=71,cinder:UPDATE=21,cinder:INSERT=7,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:13.755401 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-3d0fc42a-c477-4d73-a429-438158b15d86 req-15f68470-81f6-48e2-8052-5c8cdacc15cd tempest-ServersTestBootFromVolume-1362999175 None] attachment_update completed successfully. Sep 27 10:03:17.139682 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 req-e704eea4-b1b0-4875-b31b-c9a7d00579f8 tempest-LiveMigrationTest-1179050027 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:03:18.287271 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 req-e704eea4-b1b0-4875-b31b-c9a7d00579f8 tempest-LiveMigrationTest-1179050027 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.147s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:03:18.289448 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 req-e704eea4-b1b0-4875-b31b-c9a7d00579f8 tempest-LiveMigrationTest-1179050027 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:03:18.346096 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 req-e704eea4-b1b0-4875-b31b-c9a7d00579f8 tempest-LiveMigrationTest-1179050027 None] Connection info returned from driver {'name': 'volumes/volume-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '561adbfb-afbf-4dad-8b42-adf6941a1a02'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:03:18.470278 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 req-e704eea4-b1b0-4875-b31b-c9a7d00579f8 tempest-LiveMigrationTest-1179050027 None] attachment_update completed successfully. Sep 27 10:03:21.647999 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] Acquiring lock "cinder-814c3373-89e6-4c1a-b1e0-8ddb0523d638-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:03:21.677767 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] Lock "cinder-814c3373-89e6-4c1a-b1e0-8ddb0523d638-delete_volume" acquired by "delete_volume" :: waited 0.030s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:03:21.757592 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:03:21.889544 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:03:21.890036 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] Volume volume-814c3373-89e6-4c1a-b1e0-8ddb0523d638 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:03:21.911141 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] deleting rbd volume volume-814c3373-89e6-4c1a-b1e0-8ddb0523d638 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:03:22.460786 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=70,cinder:INSERT=17,cinder:UPDATE=12,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:22.533134 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] Created reservations ['7f67486f-2296-49ef-ab76-26eb98685e6c', 'e131d52d-0438-4315-819a-3faef7829253', '84f6b3bf-1201-43ac-9246-94b534ac7deb', '16076beb-0eb0-4d71-82f8-926893cd4bc8'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:03:22.806269 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:03:22.879190 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:03:22.973348 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:03:22.985067 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] Deleted volume successfully. Sep 27 10:03:23.013965 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-89fd9208-c783-4333-ad5b-c9221c638d86 tempest-AttachSCSIVolumeTestJSON-884590422 None] Lock "cinder-814c3373-89e6-4c1a-b1e0-8ddb0523d638-delete_volume" released by "delete_volume" :: held 1.336s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:03:29.692415 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:03:29.694866 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:03:29.783023 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:03:29.861631 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:03:32.997337 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=7,cinder:SELECT=2,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:34.202451 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 req-df1465d2-fa66-4f21-91ab-43056b8ddaf6 tempest-LiveMigrationTest-1179050027 None] Terminate volume connection completed successfully. Sep 27 10:03:34.203117 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 req-df1465d2-fa66-4f21-91ab-43056b8ddaf6 tempest-LiveMigrationTest-1179050027 None] Deleting attachment 90f296cc-7fc0-42b9-a47b-ea9dcd3bf47b. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:03:37.709537 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:03:37.733838 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Flow 'volume_create_manager' (088fbed2-8893-4219-ae9b-7898d0106255) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:03:37.740948 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4e87a7b7-ec53-4fbb-aa8f-b59f006686be) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:37.759714 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4e87a7b7-ec53-4fbb-aa8f-b59f006686be) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:03:37Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveAutoBlockMigrationV225Test-volume-106054662',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3791482b-9549-44ab-8cae-118f99dd9b3e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2369f5ee467643948f618bbdee837521',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:03:38Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:03:38Z,use_quota=True,user_id='29d2f7b0e8274c769521d13930be9825',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:37.766300 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e0add732-ffdb-4498-97bc-62c2f45b2173) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:37.771438 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e0add732-ffdb-4498-97bc-62c2f45b2173) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:37.777754 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dea7b26a-360f-41cc-9f6b-0b9b8b1c4873) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:37.781724 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dea7b26a-360f-41cc-9f6b-0b9b8b1c4873) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '3791482b-9549-44ab-8cae-118f99dd9b3e', 'volume_name': 'volume-3791482b-9549-44ab-8cae-118f99dd9b3e', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:37.787485 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2e523ac2-2ce5-46cc-bf76-88e1e3071cb3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:37.791913 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2e523ac2-2ce5-46cc-bf76-88e1e3071cb3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:37.797111 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (586f687f-3ba0-4d88-b0c0-5e50f16a031c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:37.798547 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Volume 3791482b-9549-44ab-8cae-118f99dd9b3e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-3791482b-9549-44ab-8cae-118f99dd9b3e', 'volume_size': 1} Sep 27 10:03:37.799173 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] creating volume 'volume-3791482b-9549-44ab-8cae-118f99dd9b3e' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:03:37.800169 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:03:37.900622 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:03:37.901027 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:03:37.904840 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (586f687f-3ba0-4d88-b0c0-5e50f16a031c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3791482b-9549-44ab-8cae-118f99dd9b3e', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:37.909417 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (418b5195-38fd-4bc0-8b12-58f3b147167b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:37.921791 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Volume volume-3791482b-9549-44ab-8cae-118f99dd9b3e (3791482b-9549-44ab-8cae-118f99dd9b3e): created successfully Sep 27 10:03:37.927736 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (418b5195-38fd-4bc0-8b12-58f3b147167b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:37.935346 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Flow 'volume_create_manager' (088fbed2-8893-4219-ae9b-7898d0106255) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:03:37.949749 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5d4029eb-7565-4828-aa0c-34cc76daa764 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Created volume successfully. Sep 27 10:03:40.550356 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 req-2c495571-b31f-45aa-882a-ede846ccc1a0 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:03:41.991428 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 req-2c495571-b31f-45aa-882a-ede846ccc1a0 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.440s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:03:41.993109 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 req-2c495571-b31f-45aa-882a-ede846ccc1a0 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-3791482b-9549-44ab-8cae-118f99dd9b3e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '3791482b-9549-44ab-8cae-118f99dd9b3e', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:03:42.027453 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 req-2c495571-b31f-45aa-882a-ede846ccc1a0 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Connection info returned from driver {'name': 'volumes/volume-3791482b-9549-44ab-8cae-118f99dd9b3e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '3791482b-9549-44ab-8cae-118f99dd9b3e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '7fce2555-945f-4f20-a5ab-7d2689de1443'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:03:42.145481 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 req-2c495571-b31f-45aa-882a-ede846ccc1a0 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] attachment_update completed successfully. Sep 27 10:03:44.793051 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=52,cinder:UPDATE=17,cinder:INSERT=4,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:47.376249 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-7659be88-4874-4490-ad0d-1a5f65acbfa9 req-303a10f1-c95c-47c9-adb1-76c1e5daa2f0 tempest-LiveMigrationTest-1517090534 None] Terminate volume connection completed successfully. Sep 27 10:03:47.376836 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-7659be88-4874-4490-ad0d-1a5f65acbfa9 req-303a10f1-c95c-47c9-adb1-76c1e5daa2f0 tempest-LiveMigrationTest-1517090534 None] Deleting attachment 561adbfb-afbf-4dad-8b42-adf6941a1a02. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:03:49.636095 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:03:49.662236 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Flow 'volume_create_manager' (1b6e0347-4cd8-4243-8667-15a064a01307) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:03:49.670708 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0d4b1023-f8f0-4ba5-9aac-009ac26d2dfd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:49.692982 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0d4b1023-f8f0-4ba5-9aac-009ac26d2dfd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:03:49Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1204959061',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=51c64884-725f-4278-8118-f50cc398b48a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ec0980b5824b425fbad9b8554cf5c42c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:03:50Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:03:50Z,use_quota=True,user_id='dce01d3212934161a38be853b4cd93ae',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:49.700153 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (34d952d4-3a84-49f1-9a6f-4d645c81268c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:49.705893 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (34d952d4-3a84-49f1-9a6f-4d645c81268c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:49.712870 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5b814f5e-88e9-4a01-80a7-0049e09675b2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:49.839885 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5b814f5e-88e9-4a01-80a7-0049e09675b2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '51c64884-725f-4278-8118-f50cc398b48a', 'volume_name': 'volume-51c64884-725f-4278-8118-f50cc398b48a', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:49.845490 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c1054a31-0f53-48fa-8530-f4d2f6287d29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:49.850067 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c1054a31-0f53-48fa-8530-f4d2f6287d29) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:49.854875 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2b3204d3-62dd-422f-a01d-22d5ab35d5ea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:49.856358 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Volume 51c64884-725f-4278-8118-f50cc398b48a: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-51c64884-725f-4278-8118-f50cc398b48a', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:03:49.857303 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Cloning 51c64884-725f-4278-8118-f50cc398b48a from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:03:49.858266 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:03:49.875697 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.017s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:03:49.889032 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 3, 49, 880877)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:03:49.889793 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:03:49.890512 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:03:49.900749 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.023s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:03:49.904011 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:03:49.913316 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 3, 49, 910561)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:03:49.914114 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:03:49.914768 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:03:49.933148 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:03:50.011142 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] creating snapshot='volume-51c64884-725f-4278-8118-f50cc398b48a.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:03:51.101403 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-51c64884-725f-4278-8118-f50cc398b48a.clone_snap' to 'volume-51c64884-725f-4278-8118-f50cc398b48a' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:03:51.300139 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:03:51.303334 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:03:51.303900 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:03:51.339465 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:03:51.340508 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Marking volume 51c64884-725f-4278-8118-f50cc398b48a as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:03:51.351386 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 51c64884-725f-4278-8118-f50cc398b48a. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:03:51.351872 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Creating volume glance metadata for volume 51c64884-725f-4278-8118-f50cc398b48a backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:03:51.442612 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2b3204d3-62dd-422f-a01d-22d5ab35d5ea) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-51c64884-725f-4278-8118-f50cc398b48a', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:51.448544 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (59c38e4c-54d8-4e2a-b032-b58567c4a300) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:03:51.468995 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Volume volume-51c64884-725f-4278-8118-f50cc398b48a (51c64884-725f-4278-8118-f50cc398b48a): created successfully Sep 27 10:03:51.475019 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (59c38e4c-54d8-4e2a-b032-b58567c4a300) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:03:51.484722 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Flow 'volume_create_manager' (1b6e0347-4cd8-4243-8667-15a064a01307) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:03:51.503349 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ed61ceb3-a581-493f-91cb-16d5435a2c3e tempest-LiveMigrationTest-1517090534 None] Created volume successfully. Sep 27 10:03:53.256042 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-4c19f787-4624-4839-8912-fb63c53cbcf0 req-8849760d-4939-4cc5-8dc0-8ff699b96b85 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:03:54.171612 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-4c19f787-4624-4839-8912-fb63c53cbcf0 req-8849760d-4939-4cc5-8dc0-8ff699b96b85 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.915s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:03:54.172473 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-4c19f787-4624-4839-8912-fb63c53cbcf0 req-8849760d-4939-4cc5-8dc0-8ff699b96b85 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-3791482b-9549-44ab-8cae-118f99dd9b3e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '3791482b-9549-44ab-8cae-118f99dd9b3e', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:03:54.231373 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-4c19f787-4624-4839-8912-fb63c53cbcf0 req-8849760d-4939-4cc5-8dc0-8ff699b96b85 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] Connection info returned from driver {'name': 'volumes/volume-3791482b-9549-44ab-8cae-118f99dd9b3e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '3791482b-9549-44ab-8cae-118f99dd9b3e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '70c013db-fbb0-4d0c-8313-9124d3ff777c'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:03:54.322978 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-4c19f787-4624-4839-8912-fb63c53cbcf0 req-8849760d-4939-4cc5-8dc0-8ff699b96b85 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] attachment_update completed successfully. Sep 27 10:03:59.840327 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f req-787876e1-8a0c-42f9-94c5-211671c030e0 tempest-LiveMigrationTest-1517090534 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:04:01.423687 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f req-787876e1-8a0c-42f9-94c5-211671c030e0 tempest-LiveMigrationTest-1517090534 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.583s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:04:01.424451 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f req-787876e1-8a0c-42f9-94c5-211671c030e0 tempest-LiveMigrationTest-1517090534 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-51c64884-725f-4278-8118-f50cc398b48a', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '51c64884-725f-4278-8118-f50cc398b48a', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:04:01.494046 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f req-787876e1-8a0c-42f9-94c5-211671c030e0 tempest-LiveMigrationTest-1517090534 None] Connection info returned from driver {'name': 'volumes/volume-51c64884-725f-4278-8118-f50cc398b48a', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '51c64884-725f-4278-8118-f50cc398b48a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'ca299484-709d-44d2-a53b-408f8d3d290f'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:04:01.679465 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f req-787876e1-8a0c-42f9-94c5-211671c030e0 tempest-LiveMigrationTest-1517090534 None] attachment_update completed successfully. Sep 27 10:04:04.782295 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=69,cinder:UPDATE=10,cinder:INSERT=7,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:08.719463 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-4c19f787-4624-4839-8912-fb63c53cbcf0 req-ea2f41d7-7174-4a16-be47-3a1da5d9cac7 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] Terminate volume connection completed successfully. Sep 27 10:04:08.722447 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-4c19f787-4624-4839-8912-fb63c53cbcf0 req-ea2f41d7-7174-4a16-be47-3a1da5d9cac7 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] Deleting attachment 7fce2555-945f-4f20-a5ab-7d2689de1443. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:04:19.739177 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=37,cinder:UPDATE=8,cinder:INSERT=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:19.783759 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 req-15b1e7d1-1bf7-4402-92ad-a6f242072d52 tempest-LiveMigrationTest-1179050027 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:04:20.717343 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-177a58a9-4520-4085-a36c-7f644f3ff2de req-b7d6cf77-d99d-4ab6-ab13-d27d9583c17a tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Terminate volume connection completed successfully. Sep 27 10:04:20.720321 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-177a58a9-4520-4085-a36c-7f644f3ff2de req-b7d6cf77-d99d-4ab6-ab13-d27d9583c17a tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Deleting attachment 70c013db-fbb0-4d0c-8313-9124d3ff777c. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:04:21.330866 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 req-15b1e7d1-1bf7-4402-92ad-a6f242072d52 tempest-LiveMigrationTest-1179050027 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.547s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:04:21.332604 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 req-15b1e7d1-1bf7-4402-92ad-a6f242072d52 tempest-LiveMigrationTest-1179050027 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-51c64884-725f-4278-8118-f50cc398b48a', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '51c64884-725f-4278-8118-f50cc398b48a', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:04:21.422436 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 req-15b1e7d1-1bf7-4402-92ad-a6f242072d52 tempest-LiveMigrationTest-1179050027 None] Connection info returned from driver {'name': 'volumes/volume-51c64884-725f-4278-8118-f50cc398b48a', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '51c64884-725f-4278-8118-f50cc398b48a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'e00a5985-2dda-43e1-bf02-d5fe0855dfc0'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:04:21.595600 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 req-15b1e7d1-1bf7-4402-92ad-a6f242072d52 tempest-LiveMigrationTest-1179050027 None] attachment_update completed successfully. Sep 27 10:04:24.282346 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:04:24.348107 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Flow 'volume_create_manager' (b0a88cdf-357f-4595-a2e4-931705da512a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:04:24.378312 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (398320b1-4e54-4392-945b-a8db4c2056d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:04:24.476419 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (398320b1-4e54-4392-945b-a8db4c2056d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:04:24Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1999795725',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2369f5ee467643948f618bbdee837521',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:04:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:04:24Z,use_quota=True,user_id='29d2f7b0e8274c769521d13930be9825',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:04:24.497293 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (731d40b1-a4ac-40ec-9a9d-8746ab0a60ae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:04:24.504495 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (731d40b1-a4ac-40ec-9a9d-8746ab0a60ae) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:04:24.548265 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bd41c036-a24a-436c-bdc9-ea49c5c67606) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:04:25.382967 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bd41c036-a24a-436c-bdc9-ea49c5c67606) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'volume_name': 'volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:04:25.404982 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (053a6cf9-a6a5-4455-9ad7-7545432978ef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:04:25.431738 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (053a6cf9-a6a5-4455-9ad7-7545432978ef) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:04:25.462641 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (764716bb-07ee-45ef-b64d-2412edd5ae1d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:04:25.468407 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Volume 9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:04:25.472933 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Cloning 9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5 from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:04:25.474324 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:04:25.514757 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.040s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:04:25.533216 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 4, 25, 520412)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:04:25.536324 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:04:25.537844 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:04:25.552814 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.038s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:04:25.577261 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:04:25.588984 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 4, 25, 583974)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:04:25.592548 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:04:25.593618 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:04:25.645920 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:25.786649 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] creating snapshot='volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:04:25.947811 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5.clone_snap' to 'volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:04:26.211875 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:04:26.213810 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:04:26.214510 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:04:26.255205 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:04:26.257491 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Marking volume 9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:04:26.280895 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:04:26.281486 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Creating volume glance metadata for volume 9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5 backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:04:26.463502 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (764716bb-07ee-45ef-b64d-2412edd5ae1d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:04:26.484054 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dc4821db-4d3a-4aff-86b0-ce4f80b5c669) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:04:26.512722 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Volume volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5 (9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5): created successfully Sep 27 10:04:26.519218 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dc4821db-4d3a-4aff-86b0-ce4f80b5c669) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:04:26.549505 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Flow 'volume_create_manager' (b0a88cdf-357f-4595-a2e4-931705da512a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:04:26.577720 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-749e4ef0-1aa5-4fec-ad5d-e4f2dcfdcb7b tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Created volume successfully. Sep 27 10:04:29.697940 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:04:29.698839 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:29.752534 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:29.843885 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:04:33.479135 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 req-8c69d691-128d-454f-8da2-29174e861a42 tempest-LiveMigrationTest-1179050027 None] Terminate volume connection completed successfully. Sep 27 10:04:33.480447 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 req-8c69d691-128d-454f-8da2-29174e861a42 tempest-LiveMigrationTest-1179050027 None] Deleting attachment ca299484-709d-44d2-a53b-408f8d3d290f. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:04:33.909837 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 req-c7ceec21-4f50-41fe-b766-8cab1ef09de9 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:04:36.242892 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 req-c7ceec21-4f50-41fe-b766-8cab1ef09de9 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 2.333s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:04:36.249792 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 req-c7ceec21-4f50-41fe-b766-8cab1ef09de9 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:04:36.272060 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=50,cinder:UPDATE=6,cinder:INSERT=12,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:36.360486 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 req-c7ceec21-4f50-41fe-b766-8cab1ef09de9 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Connection info returned from driver {'name': 'volumes/volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '4ab823f0-cfe9-4c97-ae51-3392c592e6a3'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:04:36.608639 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 req-c7ceec21-4f50-41fe-b766-8cab1ef09de9 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] attachment_update completed successfully. Sep 27 10:04:52.581478 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-79eec383-7ffb-4002-aa99-3c81ac17ddf9 tempest-LiveMigrationTest-1517090534 None] Terminate volume connection completed successfully. Sep 27 10:04:52.582223 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-79eec383-7ffb-4002-aa99-3c81ac17ddf9 tempest-LiveMigrationTest-1517090534 None] Deleting attachment e00a5985-2dda-43e1-bf02-d5fe0855dfc0. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:04:52.893183 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] Acquiring lock "cinder-51c64884-725f-4278-8118-f50cc398b48a-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:04:52.921387 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] Lock "cinder-51c64884-725f-4278-8118-f50cc398b48a-delete_volume" acquired by "delete_volume" :: waited 0.028s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:04:52.942491 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=43,cinder:UPDATE=9 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:53.145294 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:53.341247 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:04:53.353296 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] deleting rbd volume volume-51c64884-725f-4278-8118-f50cc398b48a {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:04:53.690304 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-53e87612-7ec5-43b5-940a-cb816bc7ed64 tempest-ServersTestBootFromVolume-1362999175 None] Terminate volume connection completed successfully. Sep 27 10:04:53.691570 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-53e87612-7ec5-43b5-940a-cb816bc7ed64 tempest-ServersTestBootFromVolume-1362999175 None] Deleting attachment 3c2bfdba-ff96-41d9-8ac6-e242764807e4. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:04:53.941318 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:04:53.990729 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] deleting parent snapshot volume-51c64884-725f-4278-8118-f50cc398b48a.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:04:55.103177 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] Acquiring lock "cinder-ff80fef3-6188-4603-afd2-55aa0b605e0c-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:04:55.140759 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] Lock "cinder-ff80fef3-6188-4603-afd2-55aa0b605e0c-delete_volume" acquired by "delete_volume" :: waited 0.038s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:04:55.259070 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:55.321899 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:04:55.322283 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] volume has clone snapshot(s) {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1378}} Sep 27 10:04:55.323643 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:04:55.336478 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] Created reservations ['741d7649-1f31-47ee-befd-0b0b269771ce', '9be8bdfb-ef01-4a59-bd2e-37d140aa6497', '3f2d6250-3b17-4fd9-8d31-7183cebcb437', '5c65c57a-274c-4d02-b441-65288825505b'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:04:55.434264 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:55.536580 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:55.648414 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:04:55.657456 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] Acquiring lock "cinder-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:04:55.665766 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] Deleted volume successfully. Sep 27 10:04:55.694336 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] Lock "cinder-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7-delete_volume" acquired by "delete_volume" :: waited 0.036s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:04:55.732365 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-8d401e23-4139-4ea6-ab72-486bddca97b0 req-b1a278c8-ad6e-4caf-a6a6-0805c2d1d793 tempest-LiveMigrationTest-1517090534 None] Lock "cinder-51c64884-725f-4278-8118-f50cc398b48a-delete_volume" released by "delete_volume" :: held 2.812s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:04:55.795535 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:55.877329 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:04:55.877750 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] Volume volume-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:04:55.886797 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] deleting rbd volume volume-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:04:55.911932 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] Created reservations ['768c83c7-d0ce-4323-b3ee-33bb1af9061a', '883fe1b5-c3ed-4f9f-838a-15d19e1a1a1b', '32fad48e-506c-41c4-94fb-9f5a69fab431', '91226c54-3bdd-4f53-9062-283fb1d79600'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:04:56.081793 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:56.186431 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:56.293183 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:04:56.299174 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] Deleted volume successfully. Sep 27 10:04:56.329198 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a req-6e80be61-cfbe-4404-ba0a-ae1275b66f36 tempest-ServersTestBootFromVolume-1362999175 None] Lock "cinder-ff80fef3-6188-4603-afd2-55aa0b605e0c-delete_volume" released by "delete_volume" :: held 1.188s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:04:56.424362 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] Created reservations ['20b761a1-d737-4753-89e8-147a9b531614', 'ff20bc3d-f86a-42ca-a4b9-2bc7ae38cd0a', 'da1f6d56-bee0-4cbe-b13e-604f544ffc01', 'd825684e-4d8c-4109-9ae3-79fc23f22b4f'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:04:56.520949 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:56.587780 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:04:56.677495 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:04:56.682615 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] Deleted volume successfully. Sep 27 10:04:56.700178 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c15e2ca3-ebba-4c04-b26d-5872371ae24b tempest-LiveMigrationTest-1517090534 None] Lock "cinder-ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7-delete_volume" released by "delete_volume" :: held 1.007s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:05:05.006887 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-329f59b7-93ec-4914-bc7d-a3d827dcd126 req-88163818-cb51-4fff-80e7-e000d1a68ed2 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:05:06.911193 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-329f59b7-93ec-4914-bc7d-a3d827dcd126 req-88163818-cb51-4fff-80e7-e000d1a68ed2 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.904s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:05:06.913105 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-329f59b7-93ec-4914-bc7d-a3d827dcd126 req-88163818-cb51-4fff-80e7-e000d1a68ed2 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:05:06.957299 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=59,cinder:UPDATE=21,cinder:INSERT=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:06.966314 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-329f59b7-93ec-4914-bc7d-a3d827dcd126 req-88163818-cb51-4fff-80e7-e000d1a68ed2 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] Connection info returned from driver {'name': 'volumes/volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'a6391459-9d27-493c-9367-1dad905325ed'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:05:07.163294 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-329f59b7-93ec-4914-bc7d-a3d827dcd126 req-88163818-cb51-4fff-80e7-e000d1a68ed2 tempest-LiveAutoBlockMigrationV225Test-1224109597 None] attachment_update completed successfully. Sep 27 10:05:20.053122 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-329f59b7-93ec-4914-bc7d-a3d827dcd126 req-3291a60b-7ff2-4fae-b898-9d662b1e385e tempest-LiveAutoBlockMigrationV225Test-1224109597 None] Terminate volume connection completed successfully. Sep 27 10:05:20.054557 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-329f59b7-93ec-4914-bc7d-a3d827dcd126 req-3291a60b-7ff2-4fae-b898-9d662b1e385e tempest-LiveAutoBlockMigrationV225Test-1224109597 None] Deleting attachment 4ab823f0-cfe9-4c97-ae51-3392c592e6a3. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:05:25.107996 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=57,cinder:UPDATE=32,cinder:INSERT=7,cinder:DELETE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:29.699422 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:05:29.700605 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:05:29.794272 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:05:29.899367 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:05:36.268879 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-77d8d0fc-7e85-4270-8da4-9850e82ecf1d tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Terminate volume connection completed successfully. Sep 27 10:05:36.269488 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-77d8d0fc-7e85-4270-8da4-9850e82ecf1d tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Deleting attachment a6391459-9d27-493c-9367-1dad905325ed. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:05:36.518604 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Acquiring lock "cinder-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:05:36.564207 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Lock "cinder-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5-delete_volume" acquired by "delete_volume" :: waited 0.046s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:05:36.655865 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:05:36.753326 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:05:36.767156 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] deleting rbd volume volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:05:37.162556 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:05:37.209253 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] deleting parent snapshot volume-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:05:38.316858 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=14,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:38.327470 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Acquiring lock "cinder-3791482b-9549-44ab-8cae-118f99dd9b3e-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:05:38.396896 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Lock "cinder-3791482b-9549-44ab-8cae-118f99dd9b3e-delete_volume" acquired by "delete_volume" :: waited 0.067s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:05:38.472339 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Created reservations ['5f451980-24b7-46b6-a65d-aff46487ebaa', '59c67502-a48c-42a4-9e26-0d0802eaa693', '5006aedf-c8aa-477f-a360-b31b7aa329cf', 'd3de2385-bd21-4f81-8c7d-b0326f5a94f5'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:05:38.509901 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:05:38.580242 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:05:38.581029 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Volume volume-3791482b-9549-44ab-8cae-118f99dd9b3e is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:05:38.591570 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] deleting rbd volume volume-3791482b-9549-44ab-8cae-118f99dd9b3e {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:05:38.634846 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:05:38.697621 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:05:38.768775 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:05:38.773142 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Deleted volume successfully. Sep 27 10:05:38.793346 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f req-b05cef8d-2b85-4e8b-9894-349cfe51f372 tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Lock "cinder-9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5-delete_volume" released by "delete_volume" :: held 2.229s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:05:38.930521 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Created reservations ['272f0b01-3326-45ff-bac1-8c903dcd3935', 'ecc03d86-f463-4fba-b06f-c3336736357a', 'dbfb3569-e9f8-46db-8fab-973f8c8b16a8', '36cb1a7d-fe49-494f-8651-10124cd0d278'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:05:39.042177 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:05:39.103866 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:05:39.164882 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:05:39.169453 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Deleted volume successfully. Sep 27 10:05:39.184187 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e7ec10f4-1bf5-4136-9868-af41d51a13db tempest-LiveAutoBlockMigrationV225Test-1036261617 None] Lock "cinder-3791482b-9549-44ab-8cae-118f99dd9b3e-delete_volume" released by "delete_volume" :: held 0.789s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:05:49.175237 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=20,cinder:INSERT=4,cinder:UPDATE=18,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:55.111962 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=29,cinder:UPDATE=15,cinder:INSERT=4,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:05.125688 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:15.115750 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:29.705504 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:06:29.706201 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:06:29.826017 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:06:29.898623 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:06:31.794642 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:06:31.816678 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Flow 'volume_create_manager' (99148fa4-9b03-420c-9fa3-068cf7f8907e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:06:31.824055 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cb849c03-5a9f-4b24-b31d-0969cb054f54) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:06:31.842417 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cb849c03-5a9f-4b24-b31d-0969cb054f54) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:06:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-DeleteServersTestJSON-volume-1226851286',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3f763a50-6adb-4ae4-8f2a-2599d21fe850,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6c681ff8fe354f97abb5fe0d5181e6f1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:06:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:06:32Z,use_quota=True,user_id='7f7c5adbac4f48b6bc141e0f888f2496',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:06:31.847909 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8a152aba-509b-473b-a6b1-9a3b1d8a1648) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:06:31.852224 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8a152aba-509b-473b-a6b1-9a3b1d8a1648) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:06:31.857826 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0e888e00-8ce1-409c-aa3a-360dc7926c73) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:06:31.862068 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0e888e00-8ce1-409c-aa3a-360dc7926c73) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '3f763a50-6adb-4ae4-8f2a-2599d21fe850', 'volume_name': 'volume-3f763a50-6adb-4ae4-8f2a-2599d21fe850', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:06:31.867605 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (778fc688-29fc-4d04-9a70-b9f03a479fb9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:06:31.871515 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (778fc688-29fc-4d04-9a70-b9f03a479fb9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:06:31.876061 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aa1b481c-a7e0-42ea-90d9-a2fa190d3aef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:06:31.877265 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Volume 3f763a50-6adb-4ae4-8f2a-2599d21fe850: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-3f763a50-6adb-4ae4-8f2a-2599d21fe850', 'volume_size': 1} Sep 27 10:06:31.877734 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] creating volume 'volume-3f763a50-6adb-4ae4-8f2a-2599d21fe850' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:06:31.878754 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:06:32.025767 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:06:32.026199 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:06:32.029934 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aa1b481c-a7e0-42ea-90d9-a2fa190d3aef) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3f763a50-6adb-4ae4-8f2a-2599d21fe850', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:06:32.034369 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4a0d2212-8dbd-4386-bed0-8efd12981b20) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:06:32.044575 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Volume volume-3f763a50-6adb-4ae4-8f2a-2599d21fe850 (3f763a50-6adb-4ae4-8f2a-2599d21fe850): created successfully Sep 27 10:06:32.052033 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4a0d2212-8dbd-4386-bed0-8efd12981b20) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:06:32.060238 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Flow 'volume_create_manager' (99148fa4-9b03-420c-9fa3-068cf7f8907e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:06:32.074662 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-243c9be7-06e9-485d-95ff-b13f6762e228 tempest-DeleteServersTestJSON-1957619751 None] Created volume successfully. Sep 27 10:06:34.156842 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 req-c639a296-b301-4f58-bacb-b517476facb0 tempest-DeleteServersTestJSON-1957619751 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:06:35.640971 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 req-c639a296-b301-4f58-bacb-b517476facb0 tempest-DeleteServersTestJSON-1957619751 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.484s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:06:35.641701 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 req-c639a296-b301-4f58-bacb-b517476facb0 tempest-DeleteServersTestJSON-1957619751 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-3f763a50-6adb-4ae4-8f2a-2599d21fe850', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '3f763a50-6adb-4ae4-8f2a-2599d21fe850', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:06:35.644618 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=13,cinder:UPDATE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:35.693662 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 req-c639a296-b301-4f58-bacb-b517476facb0 tempest-DeleteServersTestJSON-1957619751 None] Connection info returned from driver {'name': 'volumes/volume-3f763a50-6adb-4ae4-8f2a-2599d21fe850', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '3f763a50-6adb-4ae4-8f2a-2599d21fe850', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '63321a9a-d070-4d29-b42f-781e17d86cce'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:06:35.830154 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 req-c639a296-b301-4f58-bacb-b517476facb0 tempest-DeleteServersTestJSON-1957619751 None] attachment_update completed successfully. Sep 27 10:06:43.247989 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-100e5a37-bc4d-408f-b772-b4d671985258 req-91e661cc-68a0-4055-9868-8d587ef38bbd tempest-DeleteServersTestJSON-1957619751 None] Terminate volume connection completed successfully. Sep 27 10:06:43.249418 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-100e5a37-bc4d-408f-b772-b4d671985258 req-91e661cc-68a0-4055-9868-8d587ef38bbd tempest-DeleteServersTestJSON-1957619751 None] Deleting attachment 63321a9a-d070-4d29-b42f-781e17d86cce. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:06:45.142277 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=25,cinder:UPDATE=3,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:05.116555 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:05.128023 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:25.126276 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:29.705275 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:07:29.706369 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:07:29.755699 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:07:29.812341 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:07:32.016759 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:07:32.054247 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Flow 'volume_create_manager' (8dc01898-02ec-4949-a066-c0230c250ab0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:07:32.064068 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3107bc23-a5b6-4daf-9633-2c808917551b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:32.101307 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3107bc23-a5b6-4daf-9633-2c808917551b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:07:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServersNegativeTestJSON-volume-1231454589',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6696cf01-1e97-4497-ade5-48bf358e8e2d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='027612746ecc44f4ab8e7af121ec27a1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:07:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:07:32Z,use_quota=True,user_id='92b0186cd92744dabd001b86560b5c90',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:32.107525 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (af9acde4-fae2-44f9-9bc7-a65431216668) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:32.116150 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (af9acde4-fae2-44f9-9bc7-a65431216668) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:32.122260 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f630bea6-b808-490f-8370-761321f4dcb9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:32.126768 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f630bea6-b808-490f-8370-761321f4dcb9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6696cf01-1e97-4497-ade5-48bf358e8e2d', 'volume_name': 'volume-6696cf01-1e97-4497-ade5-48bf358e8e2d', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:32.134068 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1b5a8695-156d-4e36-ae91-f50d06ad18a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:32.139543 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1b5a8695-156d-4e36-ae91-f50d06ad18a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:32.145961 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7e5678f5-183d-40bc-839b-29819d01a75e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:32.147478 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Volume 6696cf01-1e97-4497-ade5-48bf358e8e2d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6696cf01-1e97-4497-ade5-48bf358e8e2d', 'volume_size': 1} Sep 27 10:07:32.148390 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] creating volume 'volume-6696cf01-1e97-4497-ade5-48bf358e8e2d' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:07:32.149290 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:07:32.253091 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:07:32.257165 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:07:32.261338 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7e5678f5-183d-40bc-839b-29819d01a75e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6696cf01-1e97-4497-ade5-48bf358e8e2d', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:32.268950 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (46d21447-28ac-4728-b717-c8051fa45271) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:32.289351 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Volume volume-6696cf01-1e97-4497-ade5-48bf358e8e2d (6696cf01-1e97-4497-ade5-48bf358e8e2d): created successfully Sep 27 10:07:32.293699 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (46d21447-28ac-4728-b717-c8051fa45271) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:32.306412 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Flow 'volume_create_manager' (8dc01898-02ec-4949-a066-c0230c250ab0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:07:32.318115 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7f7378c0-d99d-477d-a273-cf1dfc413a04 tempest-ServersNegativeTestJSON-2136329196 None] Created volume successfully. Sep 27 10:07:35.134645 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=11,cinder:UPDATE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:35.470426 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:07:35.537532 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Flow 'volume_create_manager' (546de81e-cce7-42da-94f5-021c0a9008f3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:07:35.545554 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (81f1694b-f05f-46e5-a306-c35ad9793e13) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:35.566268 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (81f1694b-f05f-46e5-a306-c35ad9793e13) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:07:35Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServersNegativeTestJSON-volume-755403975',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=40ba1110-077d-49a8-9222-bed471d90f62,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='027612746ecc44f4ab8e7af121ec27a1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:07:35Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:07:35Z,use_quota=True,user_id='92b0186cd92744dabd001b86560b5c90',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:35.573187 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3c905335-8352-4400-958d-205c438f496f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:35.578761 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3c905335-8352-4400-958d-205c438f496f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:35.584759 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c9246aa0-0512-4ffc-8ad2-e51bec3cfa98) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:35.589941 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c9246aa0-0512-4ffc-8ad2-e51bec3cfa98) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '40ba1110-077d-49a8-9222-bed471d90f62', 'volume_name': 'volume-40ba1110-077d-49a8-9222-bed471d90f62', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:35.596504 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0126e7bb-f63d-4582-b609-8a1b8db4393e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:35.614287 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0126e7bb-f63d-4582-b609-8a1b8db4393e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:35.621286 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3c57ef23-432f-4d73-bd36-db5682417dd0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:35.622938 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Volume 40ba1110-077d-49a8-9222-bed471d90f62: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-40ba1110-077d-49a8-9222-bed471d90f62', 'volume_size': 1} Sep 27 10:07:35.623686 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] creating volume 'volume-40ba1110-077d-49a8-9222-bed471d90f62' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:07:35.624596 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:07:35.763618 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:07:35.764265 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:07:35.769065 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3c57ef23-432f-4d73-bd36-db5682417dd0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-40ba1110-077d-49a8-9222-bed471d90f62', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:35.774974 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2b7b8fa0-491d-400a-85b8-68db52247b67) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:07:35.789958 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Volume volume-40ba1110-077d-49a8-9222-bed471d90f62 (40ba1110-077d-49a8-9222-bed471d90f62): created successfully Sep 27 10:07:35.795263 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2b7b8fa0-491d-400a-85b8-68db52247b67) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:07:35.802499 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Flow 'volume_create_manager' (546de81e-cce7-42da-94f5-021c0a9008f3) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:07:35.817061 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-fab99d7a-d23b-4d9e-b7d2-5c5998ba0dfc tempest-ServersNegativeTestJSON-2136329196 None] Created volume successfully. Sep 27 10:07:55.131960 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:05.126418 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=17,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:15.166078 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:29.709465 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:08:29.711722 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:08:29.785921 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:08:29.881644 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:08:35.134717 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:45.134717 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:05.143494 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:25.148452 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:29.712615 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:09:29.713372 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:09:29.812834 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:09:29.903944 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:09:45.154861 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=9,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:54.627302 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:09:54.655615 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Flow 'volume_create_manager' (97503398-89f0-42e5-803d-23bf79a05c9f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:09:54.664110 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f6b737e1-089a-47c6-8fb1-4bb73dad4956) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:54.697571 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f6b737e1-089a-47c6-8fb1-4bb73dad4956) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:09:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-1511938215',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=52fbc488-d91e-4ac4-b7b6-28284e31df30,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f86d6a96f893464f81868717f0531bd3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:09:55Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:09:55Z,use_quota=True,user_id='04a5f2b1d0134e529f25c227954b5f20',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:54.705753 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a8b8840c-36f5-4387-9250-31b6a56216ef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:54.711769 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a8b8840c-36f5-4387-9250-31b6a56216ef) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:54.718518 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6b27186f-6e6d-40de-b249-80737ebe56be) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:54.853926 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6b27186f-6e6d-40de-b249-80737ebe56be) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '52fbc488-d91e-4ac4-b7b6-28284e31df30', 'volume_name': 'volume-52fbc488-d91e-4ac4-b7b6-28284e31df30', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:54.865174 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2c07b467-ca08-4054-b680-56e5df708164) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:54.890584 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2c07b467-ca08-4054-b680-56e5df708164) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:54.906121 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (85c5abf1-bd19-4118-b6df-326b21388669) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:54.909294 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume 52fbc488-d91e-4ac4-b7b6-28284e31df30: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-52fbc488-d91e-4ac4-b7b6-28284e31df30', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:09:54.910377 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Cloning 52fbc488-d91e-4ac4-b7b6-28284e31df30 from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:09:54.913737 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:09:54.935679 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.023s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:09:54.965950 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 9, 54, 947028)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:09:54.966641 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:09:54.967758 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:09:54.998318 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.063s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:09:54.999514 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:09:55.014073 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 9, 55, 5812)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:09:55.015760 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:09:55.017385 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:09:55.080672 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:09:55.267569 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] creating snapshot='volume-52fbc488-d91e-4ac4-b7b6-28284e31df30.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:09:56.051352 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-52fbc488-d91e-4ac4-b7b6-28284e31df30.clone_snap' to 'volume-52fbc488-d91e-4ac4-b7b6-28284e31df30' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:09:56.068016 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=3,cinder:SELECT=10 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:56.295729 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:09:56.296882 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:09:56.297251 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:09:56.342049 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:09:56.343087 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Marking volume 52fbc488-d91e-4ac4-b7b6-28284e31df30 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:09:56.361828 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 52fbc488-d91e-4ac4-b7b6-28284e31df30. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:09:56.362539 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Creating volume glance metadata for volume 52fbc488-d91e-4ac4-b7b6-28284e31df30 backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:09:56.521383 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (85c5abf1-bd19-4118-b6df-326b21388669) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-52fbc488-d91e-4ac4-b7b6-28284e31df30', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:56.545657 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (33f9c407-9a0a-4636-ab8c-91443488af01) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:56.594779 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume volume-52fbc488-d91e-4ac4-b7b6-28284e31df30 (52fbc488-d91e-4ac4-b7b6-28284e31df30): created successfully Sep 27 10:09:56.599362 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (33f9c407-9a0a-4636-ab8c-91443488af01) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:56.630758 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Flow 'volume_create_manager' (97503398-89f0-42e5-803d-23bf79a05c9f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:09:56.654307 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b0bcf872-ce6a-4735-970c-640d4f0146e3 tempest-TaggedBootDevicesTest_v242-1873294123 None] Created volume successfully. Sep 27 10:09:57.508974 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:09:57.553036 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Flow 'volume_create_manager' (3ffaeb33-565e-499f-8320-54ae451e61a6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:09:57.567553 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9f4354bb-b6b6-4805-bc47-1f2325e8668c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:57.607151 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9f4354bb-b6b6-4805-bc47-1f2325e8668c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:09:57Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-385655194',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1454250c-f356-4053-83eb-5c69c0a805d7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f86d6a96f893464f81868717f0531bd3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:09:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:09:57Z,use_quota=True,user_id='04a5f2b1d0134e529f25c227954b5f20',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:57.620635 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e13b6510-2cf1-4929-80d3-f59dd1701a51) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:57.624867 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e13b6510-2cf1-4929-80d3-f59dd1701a51) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:57.661599 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0e5ab1d1-0655-428d-9851-8623eaad1b6f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:57.666146 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0e5ab1d1-0655-428d-9851-8623eaad1b6f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1454250c-f356-4053-83eb-5c69c0a805d7', 'volume_name': 'volume-1454250c-f356-4053-83eb-5c69c0a805d7', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:57.680479 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cc7df58a-2f2d-4436-8d38-0896c9b76b15) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:57.686619 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cc7df58a-2f2d-4436-8d38-0896c9b76b15) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:57.693959 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8b8b5857-e820-47c9-a566-36953a32a2a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:57.695576 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume 1454250c-f356-4053-83eb-5c69c0a805d7: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1454250c-f356-4053-83eb-5c69c0a805d7', 'volume_size': 1} Sep 27 10:09:57.696522 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] creating volume 'volume-1454250c-f356-4053-83eb-5c69c0a805d7' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:09:57.697676 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:09:57.861358 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:09:57.861917 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:09:57.866859 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8b8b5857-e820-47c9-a566-36953a32a2a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1454250c-f356-4053-83eb-5c69c0a805d7', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:57.874820 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (403a1dac-b188-4d14-ab8e-6ac603f4f25d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:57.901075 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume volume-1454250c-f356-4053-83eb-5c69c0a805d7 (1454250c-f356-4053-83eb-5c69c0a805d7): created successfully Sep 27 10:09:57.904875 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (403a1dac-b188-4d14-ab8e-6ac603f4f25d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:57.922583 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Flow 'volume_create_manager' (3ffaeb33-565e-499f-8320-54ae451e61a6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:09:57.941573 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-be1098ff-df34-48bb-93a5-789b15c16353 tempest-TaggedBootDevicesTest_v242-1873294123 None] Created volume successfully. Sep 27 10:09:59.331331 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:09:59.398795 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Flow 'volume_create_manager' (0a781847-c90a-4baf-a5bc-f8004707f746) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:09:59.422967 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ca3dbbc6-70a4-4f17-95ce-d03aa350523b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:59.464165 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ca3dbbc6-70a4-4f17-95ce-d03aa350523b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:09:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-971106948',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1caf2529-26ae-48b5-83d4-b87bb80b3024,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f86d6a96f893464f81868717f0531bd3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:09:59Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:09:59Z,use_quota=True,user_id='04a5f2b1d0134e529f25c227954b5f20',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:59.480186 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c2a28757-cf08-4309-b6ca-0c8aa010d1dc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:59.484508 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c2a28757-cf08-4309-b6ca-0c8aa010d1dc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:59.503405 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (36a518b2-5e3d-4758-8655-7bc78249561a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:59.515870 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (36a518b2-5e3d-4758-8655-7bc78249561a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1caf2529-26ae-48b5-83d4-b87bb80b3024', 'volume_name': 'volume-1caf2529-26ae-48b5-83d4-b87bb80b3024', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:59.529833 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e0934103-52b7-4610-aced-3f1ad8f43967) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:59.542745 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e0934103-52b7-4610-aced-3f1ad8f43967) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:59.547893 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (474b15d6-c503-491b-aefb-31d980efab02) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:59.551578 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume 1caf2529-26ae-48b5-83d4-b87bb80b3024: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1caf2529-26ae-48b5-83d4-b87bb80b3024', 'volume_size': 1} Sep 27 10:09:59.551993 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] creating volume 'volume-1caf2529-26ae-48b5-83d4-b87bb80b3024' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:09:59.566323 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:09:59.722870 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:09:59.723249 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:09:59.726931 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (474b15d6-c503-491b-aefb-31d980efab02) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1caf2529-26ae-48b5-83d4-b87bb80b3024', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:59.734530 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (02664369-6b4e-4d1a-a1e0-5c11af93aaa4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:09:59.754610 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume volume-1caf2529-26ae-48b5-83d4-b87bb80b3024 (1caf2529-26ae-48b5-83d4-b87bb80b3024): created successfully Sep 27 10:09:59.759969 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (02664369-6b4e-4d1a-a1e0-5c11af93aaa4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:09:59.774082 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Flow 'volume_create_manager' (0a781847-c90a-4baf-a5bc-f8004707f746) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:09:59.793848 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d49d4ffe-7693-48da-854c-3a5b67a899c1 tempest-TaggedBootDevicesTest_v242-1873294123 None] Created volume successfully. Sep 27 10:10:16.062504 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=26,cinder:UPDATE=6,cinder:INSERT=8,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:29.714597 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:10:29.715352 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:10:29.780194 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:10:29.858411 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:10:31.295406 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=23,cinder:UPDATE=9,cinder:INSERT=4,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:31.341599 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-debfbf37-3d48-4a17-a5c1-9952098f3865 tempest-TaggedBootDevicesTest_v242-1873294123 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:10:33.063699 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-debfbf37-3d48-4a17-a5c1-9952098f3865 tempest-TaggedBootDevicesTest_v242-1873294123 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.722s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:10:33.064425 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-debfbf37-3d48-4a17-a5c1-9952098f3865 tempest-TaggedBootDevicesTest_v242-1873294123 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-52fbc488-d91e-4ac4-b7b6-28284e31df30', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '52fbc488-d91e-4ac4-b7b6-28284e31df30', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:10:33.108768 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-debfbf37-3d48-4a17-a5c1-9952098f3865 tempest-TaggedBootDevicesTest_v242-1873294123 None] Connection info returned from driver {'name': 'volumes/volume-52fbc488-d91e-4ac4-b7b6-28284e31df30', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '52fbc488-d91e-4ac4-b7b6-28284e31df30', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '957664a1-d47c-4954-a4a1-236b185addb4'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:10:33.220287 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-debfbf37-3d48-4a17-a5c1-9952098f3865 tempest-TaggedBootDevicesTest_v242-1873294123 None] attachment_update completed successfully. Sep 27 10:10:34.107645 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-55bd920b-eaa7-4c05-b34e-b2fc21bdb0a8 tempest-TaggedBootDevicesTest_v242-1873294123 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:10:35.335643 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-55bd920b-eaa7-4c05-b34e-b2fc21bdb0a8 tempest-TaggedBootDevicesTest_v242-1873294123 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.227s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:10:35.336923 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-55bd920b-eaa7-4c05-b34e-b2fc21bdb0a8 tempest-TaggedBootDevicesTest_v242-1873294123 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1454250c-f356-4053-83eb-5c69c0a805d7', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '1454250c-f356-4053-83eb-5c69c0a805d7', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:10:35.437417 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-55bd920b-eaa7-4c05-b34e-b2fc21bdb0a8 tempest-TaggedBootDevicesTest_v242-1873294123 None] Connection info returned from driver {'name': 'volumes/volume-1454250c-f356-4053-83eb-5c69c0a805d7', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '1454250c-f356-4053-83eb-5c69c0a805d7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '6866de5d-88ff-4aa7-87c0-95a92e6488bf'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:10:35.601505 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-55bd920b-eaa7-4c05-b34e-b2fc21bdb0a8 tempest-TaggedBootDevicesTest_v242-1873294123 None] attachment_update completed successfully. Sep 27 10:10:36.252165 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-5ce6072a-2304-416c-893a-6c1b684d4826 tempest-TaggedBootDevicesTest_v242-1873294123 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:10:37.373894 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-5ce6072a-2304-416c-893a-6c1b684d4826 tempest-TaggedBootDevicesTest_v242-1873294123 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.122s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:10:37.374741 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-5ce6072a-2304-416c-893a-6c1b684d4826 tempest-TaggedBootDevicesTest_v242-1873294123 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1caf2529-26ae-48b5-83d4-b87bb80b3024', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '1caf2529-26ae-48b5-83d4-b87bb80b3024', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:10:37.428511 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-5ce6072a-2304-416c-893a-6c1b684d4826 tempest-TaggedBootDevicesTest_v242-1873294123 None] Connection info returned from driver {'name': 'volumes/volume-1caf2529-26ae-48b5-83d4-b87bb80b3024', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '1caf2529-26ae-48b5-83d4-b87bb80b3024', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '58a4888d-3aa4-46d0-9007-49c5b9a251e8'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:10:37.537204 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 req-5ce6072a-2304-416c-893a-6c1b684d4826 tempest-TaggedBootDevicesTest_v242-1873294123 None] attachment_update completed successfully. Sep 27 10:10:47.532536 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=27,cinder:UPDATE=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:52.175387 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] Acquiring lock "cinder-40ba1110-077d-49a8-9222-bed471d90f62-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:10:52.205107 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] Lock "cinder-40ba1110-077d-49a8-9222-bed471d90f62-delete_volume" acquired by "delete_volume" :: waited 0.030s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:10:52.216571 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=44,cinder:UPDATE=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:52.278799 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:10:52.345510 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:10:52.346051 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] Volume volume-40ba1110-077d-49a8-9222-bed471d90f62 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:10:52.354569 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] deleting rbd volume volume-40ba1110-077d-49a8-9222-bed471d90f62 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:10:52.780462 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] Created reservations ['aec5e782-47b5-4d3a-947c-6e65cc55045b', 'd4450f55-66a6-42f1-9688-a4f0253125ef', 'e019fc48-5272-41e1-a518-6b31f9b158b3', '129d4c30-fd3d-49bd-99f9-affefb6d5921'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:10:52.847831 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:10:52.911281 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:10:52.960599 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:10:52.969125 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] Deleted volume successfully. Sep 27 10:10:52.987278 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-fbb9ebf5-3088-4c69-ab95-a2d43597e714 tempest-ServersNegativeTestJSON-2136329196 None] Lock "cinder-40ba1110-077d-49a8-9222-bed471d90f62-delete_volume" released by "delete_volume" :: held 0.782s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:10:53.468159 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] Acquiring lock "cinder-6696cf01-1e97-4497-ade5-48bf358e8e2d-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:10:53.488779 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] Lock "cinder-6696cf01-1e97-4497-ade5-48bf358e8e2d-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:10:53.543318 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:10:53.617827 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:10:53.619244 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] Volume volume-6696cf01-1e97-4497-ade5-48bf358e8e2d is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:10:53.639424 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] deleting rbd volume volume-6696cf01-1e97-4497-ade5-48bf358e8e2d {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:10:54.117469 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] Created reservations ['f47d1d40-6e8a-4002-972d-299a0bf0f7e4', '4271c21f-0f00-4e53-9d37-1c26497fde8b', '3bf52005-abb0-427d-915f-8d54b122daba', '2e403f8f-0a6d-4745-97e4-0bf3d25d9929'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:10:54.181238 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:10:54.241007 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:10:54.320973 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:10:54.326231 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] Deleted volume successfully. Sep 27 10:10:54.360635 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1e08f23a-a05f-40ae-8967-e85c0882543e tempest-ServersNegativeTestJSON-2136329196 None] Lock "cinder-6696cf01-1e97-4497-ade5-48bf358e8e2d-delete_volume" released by "delete_volume" :: held 0.874s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:11:02.958087 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] Acquiring lock "cinder-3f763a50-6adb-4ae4-8f2a-2599d21fe850-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:11:02.972322 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] Lock "cinder-3f763a50-6adb-4ae4-8f2a-2599d21fe850-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:11:03.044005 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:03.122812 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:11:03.123243 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] Volume volume-3f763a50-6adb-4ae4-8f2a-2599d21fe850 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:11:03.141403 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] deleting rbd volume volume-3f763a50-6adb-4ae4-8f2a-2599d21fe850 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:11:03.684897 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] Created reservations ['9f415bb1-ff9e-4671-90b2-cd9c912c1429', 'b7fef69a-bb9d-4f57-938f-538898bef111', '9bd91ae8-cee6-4bc6-8ac6-06b9eeec86e4', 'a2416c51-f039-4eb3-95c5-e6ea62f37177'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:11:03.791843 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:03.879132 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:03.951053 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:11:03.958463 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] Deleted volume successfully. Sep 27 10:11:03.981513 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-36fe9b97-4dc3-4d04-b9ab-fbc3f7dddd2b tempest-DeleteServersTestJSON-1957619751 None] Lock "cinder-3f763a50-6adb-4ae4-8f2a-2599d21fe850-delete_volume" released by "delete_volume" :: held 1.009s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:11:20.357509 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=47,cinder:UPDATE=30,cinder:INSERT=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:20.365124 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:11:20.390917 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Flow 'volume_create_manager' (12dc946b-0223-4edc-8555-5caf64baa989) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:20.399586 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0e7feee2-9550-4449-9a59-3517f3a4834c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:20.423340 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0e7feee2-9550-4449-9a59-3517f3a4834c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:11:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-1562829320',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b313c2a7-2f8f-4fa4-b158-8b23b1a99360,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='4dd13c6da37d4d0aa58db41f6722c823',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:11:20Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:11:20Z,use_quota=True,user_id='a12e7d80aab0420da7046cda0b553a44',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:20.429235 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (57ea9404-a799-4ce0-9410-13e0a574fb12) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:20.433476 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (57ea9404-a799-4ce0-9410-13e0a574fb12) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:20.442111 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fce83356-c0de-4bcb-b2ba-fee437d5c934) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:20.447337 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fce83356-c0de-4bcb-b2ba-fee437d5c934) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b313c2a7-2f8f-4fa4-b158-8b23b1a99360', 'volume_name': 'volume-b313c2a7-2f8f-4fa4-b158-8b23b1a99360', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:20.460867 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d82b12b9-e007-4ce4-8242-b8a4385e9525) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:20.466746 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d82b12b9-e007-4ce4-8242-b8a4385e9525) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:20.472247 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (051a026f-8c6f-4424-8f67-5ac1302c54f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:20.477440 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Volume b313c2a7-2f8f-4fa4-b158-8b23b1a99360: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b313c2a7-2f8f-4fa4-b158-8b23b1a99360', 'volume_size': 1} Sep 27 10:11:20.478047 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] creating volume 'volume-b313c2a7-2f8f-4fa4-b158-8b23b1a99360' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:11:20.479098 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:20.618997 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:11:20.619562 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:11:20.624420 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (051a026f-8c6f-4424-8f67-5ac1302c54f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b313c2a7-2f8f-4fa4-b158-8b23b1a99360', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:20.631395 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d1216a4c-a6a4-480a-b925-01d01eae0830) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:20.652201 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Volume volume-b313c2a7-2f8f-4fa4-b158-8b23b1a99360 (b313c2a7-2f8f-4fa4-b158-8b23b1a99360): created successfully Sep 27 10:11:20.657498 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d1216a4c-a6a4-480a-b925-01d01eae0830) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:20.679360 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Flow 'volume_create_manager' (12dc946b-0223-4edc-8555-5caf64baa989) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:20.702202 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ba999ef2-d01d-4d14-a43a-092eae95c77a tempest-VolumesTestJSON-50992764 None] Created volume successfully. Sep 27 10:11:22.176368 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:11:22.215121 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Flow 'volume_create_manager' (d06b90f1-3d2d-4317-bcb7-63c8131b9f04) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:22.229462 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3b1e66ae-8d53-4b26-bdc2-3c04bd287e62) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:22.254486 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3b1e66ae-8d53-4b26-bdc2-3c04bd287e62) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:11:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-1538287203',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=581dba67-f337-4542-9d7a-c4f99fb86ae0,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='4dd13c6da37d4d0aa58db41f6722c823',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:11:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:11:22Z,use_quota=True,user_id='a12e7d80aab0420da7046cda0b553a44',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:22.262197 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (705adfff-e7b9-4e69-8dad-e3e8a08712f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:22.268610 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (705adfff-e7b9-4e69-8dad-e3e8a08712f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:22.274521 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (08c19136-fa16-498a-9343-8ac47fe997d5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:22.283008 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (08c19136-fa16-498a-9343-8ac47fe997d5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '581dba67-f337-4542-9d7a-c4f99fb86ae0', 'volume_name': 'volume-581dba67-f337-4542-9d7a-c4f99fb86ae0', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:22.290829 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (42371939-2d5d-487d-bc6b-9f31ee843423) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:22.295090 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (42371939-2d5d-487d-bc6b-9f31ee843423) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:22.303315 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (42beffc1-75f6-4779-8980-4ecc6e207310) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:22.305558 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Volume 581dba67-f337-4542-9d7a-c4f99fb86ae0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-581dba67-f337-4542-9d7a-c4f99fb86ae0', 'volume_size': 1} Sep 27 10:11:22.306185 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] creating volume 'volume-581dba67-f337-4542-9d7a-c4f99fb86ae0' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:11:22.307012 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:22.469428 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:11:22.469863 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:11:22.473671 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (42beffc1-75f6-4779-8980-4ecc6e207310) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-581dba67-f337-4542-9d7a-c4f99fb86ae0', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:22.487523 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (79a92616-963c-4c74-b848-d23abce534a9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:22.491960 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=42,cinder:INSERT=7,cinder:UPDATE=23,cinder:DELETE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:22.521337 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Volume volume-581dba67-f337-4542-9d7a-c4f99fb86ae0 (581dba67-f337-4542-9d7a-c4f99fb86ae0): created successfully Sep 27 10:11:22.535158 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (79a92616-963c-4c74-b848-d23abce534a9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:22.556161 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Flow 'volume_create_manager' (d06b90f1-3d2d-4317-bcb7-63c8131b9f04) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:22.573867 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-52b231f4-ade4-4c33-9838-2ce6f7b8e1d6 tempest-VolumesTestJSON-50992764 None] Created volume successfully. Sep 27 10:11:23.903923 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:11:23.956885 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Flow 'volume_create_manager' (9477468f-99d7-42ac-b279-97e955a260fc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:23.971317 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5558183b-6e78-4d40-9d3a-16e68f461317) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:24.011585 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5558183b-6e78-4d40-9d3a-16e68f461317) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:11:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-2063678243',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a4cf7c86-4e26-462a-a552-ac303e88f53d,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='4dd13c6da37d4d0aa58db41f6722c823',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:11:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:11:24Z,use_quota=True,user_id='a12e7d80aab0420da7046cda0b553a44',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:24.024124 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a579bed7-2668-4670-a49a-66dbd1a1334f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:24.039253 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a579bed7-2668-4670-a49a-66dbd1a1334f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:24.051434 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (88284fee-9472-4457-9c2f-e93388f8bbfb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:24.068078 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (88284fee-9472-4457-9c2f-e93388f8bbfb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a4cf7c86-4e26-462a-a552-ac303e88f53d', 'volume_name': 'volume-a4cf7c86-4e26-462a-a552-ac303e88f53d', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:24.094937 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a54d091-efa4-4bd1-95ee-215ac40ffd6d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:24.112379 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a54d091-efa4-4bd1-95ee-215ac40ffd6d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:24.138798 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (17d9a436-5d38-4ae9-b359-6351d3fcb5d6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:24.142129 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Volume a4cf7c86-4e26-462a-a552-ac303e88f53d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a4cf7c86-4e26-462a-a552-ac303e88f53d', 'volume_size': 1} Sep 27 10:11:24.144723 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] creating volume 'volume-a4cf7c86-4e26-462a-a552-ac303e88f53d' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:11:24.146762 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:24.350399 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:11:24.351108 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:11:24.356815 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (17d9a436-5d38-4ae9-b359-6351d3fcb5d6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a4cf7c86-4e26-462a-a552-ac303e88f53d', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:24.364415 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5a11f7e8-437a-4f64-8598-3eb4da7da767) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:24.393661 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Volume volume-a4cf7c86-4e26-462a-a552-ac303e88f53d (a4cf7c86-4e26-462a-a552-ac303e88f53d): created successfully Sep 27 10:11:24.398818 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5a11f7e8-437a-4f64-8598-3eb4da7da767) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:24.407493 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Flow 'volume_create_manager' (9477468f-99d7-42ac-b279-97e955a260fc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:24.435690 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-bc836ce0-1f37-40ed-8efe-4cecf1eb6194 tempest-VolumesTestJSON-50992764 None] Created volume successfully. Sep 27 10:11:26.669402 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] Acquiring lock "cinder-a4cf7c86-4e26-462a-a552-ac303e88f53d-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:11:26.687926 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] Lock "cinder-a4cf7c86-4e26-462a-a552-ac303e88f53d-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:11:26.758040 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:26.874438 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:11:26.875208 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] Volume volume-a4cf7c86-4e26-462a-a552-ac303e88f53d is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:11:26.883521 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] deleting rbd volume volume-a4cf7c86-4e26-462a-a552-ac303e88f53d {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:11:27.258373 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] Created reservations ['73a4218f-06c0-4eca-8147-aacaa2babf24', 'd86d3b23-3f6c-484b-a79a-503c57362b07', 'a78f2a40-9846-4837-be71-d126659ea288', '51e3a1e8-b046-42ee-acce-47855c68e220'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:11:27.389790 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:27.477186 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:27.548142 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:11:27.558707 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] Deleted volume successfully. Sep 27 10:11:27.595241 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-359583bf-49ec-4573-9182-0cc01e657915 tempest-VolumesTestJSON-50992764 None] Lock "cinder-a4cf7c86-4e26-462a-a552-ac303e88f53d-delete_volume" released by "delete_volume" :: held 0.909s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:11:27.943505 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] Acquiring lock "cinder-581dba67-f337-4542-9d7a-c4f99fb86ae0-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:11:27.964386 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] Lock "cinder-581dba67-f337-4542-9d7a-c4f99fb86ae0-delete_volume" acquired by "delete_volume" :: waited 0.021s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:11:28.047474 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:28.149846 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:11:28.150222 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] Volume volume-581dba67-f337-4542-9d7a-c4f99fb86ae0 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:11:28.171709 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] deleting rbd volume volume-581dba67-f337-4542-9d7a-c4f99fb86ae0 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:11:28.684315 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] Created reservations ['4a23375a-6044-424e-b33d-e4886af911ac', 'fc995d00-9606-4856-b4cd-08704afec68b', '6886f360-1e06-48cf-94f3-09f0aad17f6a', '4f53d461-f0fe-4ae5-a874-626afcf91a99'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:11:28.814168 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:28.905295 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:29.008627 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:11:29.017434 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] Deleted volume successfully. Sep 27 10:11:29.044110 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8d11c291-5b6c-4429-9ec7-5c956fd4ca24 tempest-VolumesTestJSON-50992764 None] Lock "cinder-581dba67-f337-4542-9d7a-c4f99fb86ae0-delete_volume" released by "delete_volume" :: held 1.080s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:11:29.158791 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] Acquiring lock "cinder-b313c2a7-2f8f-4fa4-b158-8b23b1a99360-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:11:29.173087 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] Lock "cinder-b313c2a7-2f8f-4fa4-b158-8b23b1a99360-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:11:29.248216 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:29.327194 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:11:29.327748 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] Volume volume-b313c2a7-2f8f-4fa4-b158-8b23b1a99360 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:11:29.339673 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] deleting rbd volume volume-b313c2a7-2f8f-4fa4-b158-8b23b1a99360 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:11:29.715844 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:11:29.716532 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:29.788842 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:29.880955 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:11:29.952186 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] Created reservations ['7cf3050f-a9d6-4164-85c4-2d4e30585184', '674e52c6-ea3e-4ca3-9ba3-ce8b2aeb92a7', 'c79978c1-0910-4d5f-8189-1096cf48023e', '142eea5d-e1e1-47b7-957f-1b3033d30fde'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:11:30.094832 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:30.165576 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:30.231780 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:11:30.237363 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] Deleted volume successfully. Sep 27 10:11:30.263909 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-11896a1c-8443-4e30-b8dd-7025bec0ef21 tempest-VolumesTestJSON-50992764 None] Lock "cinder-b313c2a7-2f8f-4fa4-b158-8b23b1a99360-delete_volume" released by "delete_volume" :: held 1.091s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:11:39.188337 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:11:39.221071 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Flow 'volume_create_manager' (6f82c6ce-c8b2-4bba-8e05-91b28941a96e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:39.241969 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b3510b3a-1324-49c5-a1ea-f2e8a6d82ba8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:39.307072 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b3510b3a-1324-49c5-a1ea-f2e8a6d82ba8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:11:39Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-1011314192',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cceacac7-f31d-4e36-b764-2231c93d64eb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9c60d0e3692444e39346d2c296c32ac2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:11:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:11:39Z,use_quota=True,user_id='e4772d35d57f478d929eea70fd11e147',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:39.338739 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (69e8b63c-0075-477d-bf25-173096e1da48) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:39.352576 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (69e8b63c-0075-477d-bf25-173096e1da48) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:39.359543 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c810464b-9a76-49f6-a92b-1f279d7680c1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:39.584404 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c810464b-9a76-49f6-a92b-1f279d7680c1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'cceacac7-f31d-4e36-b764-2231c93d64eb', 'volume_name': 'volume-cceacac7-f31d-4e36-b764-2231c93d64eb', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:39.597455 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f072c374-51ef-433f-9a6a-ff58d858f29e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:39.611773 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f072c374-51ef-433f-9a6a-ff58d858f29e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:39.627509 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ab83d6fe-83c2-4e3b-82ed-d15f5d02b0bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:39.628720 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Volume cceacac7-f31d-4e36-b764-2231c93d64eb: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-cceacac7-f31d-4e36-b764-2231c93d64eb', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:11:39.636604 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Cloning cceacac7-f31d-4e36-b764-2231c93d64eb from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:11:39.638505 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:11:39.659526 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.021s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:11:39.683946 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 11, 39, 671225)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:11:39.684388 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:11:39.684875 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:11:39.690481 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.031s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:11:39.691823 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:11:39.703659 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 11, 39, 700509)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:11:39.704586 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:11:39.705279 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:11:39.710421 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=55,cinder:UPDATE=34,cinder:DELETE=2,cinder:INSERT=9 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:39.724266 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:39.844633 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] creating snapshot='volume-cceacac7-f31d-4e36-b764-2231c93d64eb.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:11:40.227471 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-cceacac7-f31d-4e36-b764-2231c93d64eb.clone_snap' to 'volume-cceacac7-f31d-4e36-b764-2231c93d64eb' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:11:40.444385 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:11:40.445593 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:11:40.445952 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:11:40.488479 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:11:40.489548 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Marking volume cceacac7-f31d-4e36-b764-2231c93d64eb as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:11:40.502104 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to cceacac7-f31d-4e36-b764-2231c93d64eb. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:11:40.502870 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Creating volume glance metadata for volume cceacac7-f31d-4e36-b764-2231c93d64eb backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:11:40.666815 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ab83d6fe-83c2-4e3b-82ed-d15f5d02b0bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cceacac7-f31d-4e36-b764-2231c93d64eb', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:40.672409 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c05b683b-9b02-48e6-8823-fd47004673e2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:40.689597 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Volume volume-cceacac7-f31d-4e36-b764-2231c93d64eb (cceacac7-f31d-4e36-b764-2231c93d64eb): created successfully Sep 27 10:11:40.695812 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c05b683b-9b02-48e6-8823-fd47004673e2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:40.724050 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Flow 'volume_create_manager' (6f82c6ce-c8b2-4bba-8e05-91b28941a96e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:40.751652 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-89bd9e3a-640f-4060-b0f1-4038b26b5fdb tempest-TaggedBootDevicesTest-1222598264 None] Created volume successfully. Sep 27 10:11:41.957573 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=60,cinder:UPDATE=27,cinder:INSERT=10,cinder:DELETE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:41.974828 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:11:42.018878 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Flow 'volume_create_manager' (fd310539-70ec-43c3-82a6-1bd05392d90e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:42.026300 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8ef219a9-b67b-473b-b8f4-0b89f16994ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:42.055092 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8ef219a9-b67b-473b-b8f4-0b89f16994ca) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:11:42Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-1393546629',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8943dda0-13e7-42e3-bdeb-f9cf11b10058,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9c60d0e3692444e39346d2c296c32ac2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:11:42Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:11:42Z,use_quota=True,user_id='e4772d35d57f478d929eea70fd11e147',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:42.064634 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (659b642e-65e2-4dbe-85be-8b8e33292026) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:42.068814 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (659b642e-65e2-4dbe-85be-8b8e33292026) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:42.074641 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1eaf0913-54fe-42ef-9e8c-acc97816e2b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:42.080957 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1eaf0913-54fe-42ef-9e8c-acc97816e2b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8943dda0-13e7-42e3-bdeb-f9cf11b10058', 'volume_name': 'volume-8943dda0-13e7-42e3-bdeb-f9cf11b10058', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:42.093296 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4dc06fac-3fc2-4ea0-91b1-11c278956fec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:42.098178 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4dc06fac-3fc2-4ea0-91b1-11c278956fec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:42.103922 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f1cf4fd5-a0c8-4e9f-aa92-eddc15f2f713) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:42.105263 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Volume 8943dda0-13e7-42e3-bdeb-f9cf11b10058: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8943dda0-13e7-42e3-bdeb-f9cf11b10058', 'volume_size': 1} Sep 27 10:11:42.105846 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] creating volume 'volume-8943dda0-13e7-42e3-bdeb-f9cf11b10058' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:11:42.107065 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:42.247391 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:11:42.248163 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:11:42.252896 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f1cf4fd5-a0c8-4e9f-aa92-eddc15f2f713) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8943dda0-13e7-42e3-bdeb-f9cf11b10058', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:42.260473 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0bf07103-646e-43b7-946a-00844e94dfc5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:42.299164 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Volume volume-8943dda0-13e7-42e3-bdeb-f9cf11b10058 (8943dda0-13e7-42e3-bdeb-f9cf11b10058): created successfully Sep 27 10:11:42.305149 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0bf07103-646e-43b7-946a-00844e94dfc5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:42.312475 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Flow 'volume_create_manager' (fd310539-70ec-43c3-82a6-1bd05392d90e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:42.329963 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-23631a02-e833-42ac-be5b-265c70ecbbf0 tempest-TaggedBootDevicesTest-1222598264 None] Created volume successfully. Sep 27 10:11:43.786664 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:11:43.852024 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Flow 'volume_create_manager' (11f57bfe-ca2e-4627-8005-b26539312a6f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:43.899753 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f5a3d532-4d23-449c-886c-00e8f8aa09c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:43.943426 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f5a3d532-4d23-449c-886c-00e8f8aa09c5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:11:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-1637191587',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4b2f9c8c-667b-4bff-8590-c93510f8acf3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9c60d0e3692444e39346d2c296c32ac2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:11:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:11:44Z,use_quota=True,user_id='e4772d35d57f478d929eea70fd11e147',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:43.956439 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b71d0563-453a-4796-8232-20d96afe7add) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:43.962571 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b71d0563-453a-4796-8232-20d96afe7add) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:43.973543 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (102e229d-96e3-42be-a4a3-21fcd7d63d65) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:43.982824 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (102e229d-96e3-42be-a4a3-21fcd7d63d65) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4b2f9c8c-667b-4bff-8590-c93510f8acf3', 'volume_name': 'volume-4b2f9c8c-667b-4bff-8590-c93510f8acf3', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:43.997937 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a918cc59-c23e-45bc-b2cd-72a1893bf60b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:44.008924 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a918cc59-c23e-45bc-b2cd-72a1893bf60b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:44.019697 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (509fc067-3577-4f38-b7e3-281239001c21) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:44.021232 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Volume 4b2f9c8c-667b-4bff-8590-c93510f8acf3: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4b2f9c8c-667b-4bff-8590-c93510f8acf3', 'volume_size': 1} Sep 27 10:11:44.024413 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] creating volume 'volume-4b2f9c8c-667b-4bff-8590-c93510f8acf3' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:11:44.025466 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:44.251884 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:11:44.252526 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:11:44.258390 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (509fc067-3577-4f38-b7e3-281239001c21) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4b2f9c8c-667b-4bff-8590-c93510f8acf3', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:44.265060 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9de36d52-45c2-44a1-a355-0fed1237304e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:44.278905 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Volume volume-4b2f9c8c-667b-4bff-8590-c93510f8acf3 (4b2f9c8c-667b-4bff-8590-c93510f8acf3): created successfully Sep 27 10:11:44.285157 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9de36d52-45c2-44a1-a355-0fed1237304e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:44.302418 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Flow 'volume_create_manager' (11f57bfe-ca2e-4627-8005-b26539312a6f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:44.320561 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-dc39d397-1a69-4dd3-9b07-400e12596089 tempest-TaggedBootDevicesTest-1222598264 None] Created volume successfully. Sep 27 10:11:46.251299 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:11:46.273831 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Flow 'volume_create_manager' (7910f74c-3093-4ce3-ba1a-bb0b719a2adb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:46.281222 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b9a9f8f4-25bc-4c0e-b133-1a5d28b6bd22) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:46.317568 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b9a9f8f4-25bc-4c0e-b133-1a5d28b6bd22) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:11:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-714894855',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f605c0e1-aee3-46fe-a624-b9b13bb6cb46,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f847bee14ff74dfbafe7c6a199f1b99f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:11:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:11:46Z,use_quota=True,user_id='b8d6f6c63fe144959139278ca287e2ec',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:46.329723 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (17aa753c-db04-4a18-9fc7-61a7c26554f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:46.358241 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (17aa753c-db04-4a18-9fc7-61a7c26554f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:46.376809 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (24258f71-4852-425b-baf1-4d6e54a610b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:46.382809 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (24258f71-4852-425b-baf1-4d6e54a610b9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f605c0e1-aee3-46fe-a624-b9b13bb6cb46', 'volume_name': 'volume-f605c0e1-aee3-46fe-a624-b9b13bb6cb46', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:46.394031 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (09fb838c-9b11-43a3-982a-2f5df93a62b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:46.420267 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (09fb838c-9b11-43a3-982a-2f5df93a62b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:46.431827 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (963d3f5e-c655-4292-8d9a-31ca0a122577) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:46.433313 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Volume f605c0e1-aee3-46fe-a624-b9b13bb6cb46: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f605c0e1-aee3-46fe-a624-b9b13bb6cb46', 'volume_size': 1} Sep 27 10:11:46.435042 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] creating volume 'volume-f605c0e1-aee3-46fe-a624-b9b13bb6cb46' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:11:46.435895 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:11:46.568168 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:11:46.568981 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:11:46.576184 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (963d3f5e-c655-4292-8d9a-31ca0a122577) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f605c0e1-aee3-46fe-a624-b9b13bb6cb46', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:46.582492 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f03fe178-8c50-4adb-8ab0-840682d7bf5b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:11:46.598257 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Volume volume-f605c0e1-aee3-46fe-a624-b9b13bb6cb46 (f605c0e1-aee3-46fe-a624-b9b13bb6cb46): created successfully Sep 27 10:11:46.606354 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f03fe178-8c50-4adb-8ab0-840682d7bf5b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:11:46.616513 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Flow 'volume_create_manager' (7910f74c-3093-4ce3-ba1a-bb0b719a2adb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:11:46.646059 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-212b0f25-e558-498b-a35c-6357a1ccb220 tempest-VolumesAdminNegativeTest-1801268990 None] Created volume successfully. Sep 27 10:11:48.244191 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-af441a36-cce9-481b-8924-2fb03ec868ff req-84f4469f-0c92-432c-b31b-a70650a15e1d tempest-VolumesAdminNegativeTest-1801268990 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:11:50.346347 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-af441a36-cce9-481b-8924-2fb03ec868ff req-84f4469f-0c92-432c-b31b-a70650a15e1d tempest-VolumesAdminNegativeTest-1801268990 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 2.101s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:11:50.348305 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-af441a36-cce9-481b-8924-2fb03ec868ff req-84f4469f-0c92-432c-b31b-a70650a15e1d tempest-VolumesAdminNegativeTest-1801268990 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f605c0e1-aee3-46fe-a624-b9b13bb6cb46', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'f605c0e1-aee3-46fe-a624-b9b13bb6cb46', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:11:50.403781 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-af441a36-cce9-481b-8924-2fb03ec868ff req-84f4469f-0c92-432c-b31b-a70650a15e1d tempest-VolumesAdminNegativeTest-1801268990 None] Connection info returned from driver {'name': 'volumes/volume-f605c0e1-aee3-46fe-a624-b9b13bb6cb46', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'f605c0e1-aee3-46fe-a624-b9b13bb6cb46', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'b58224c9-3fc9-483f-8b43-85dc37cb507f'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:11:50.561576 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-af441a36-cce9-481b-8924-2fb03ec868ff req-84f4469f-0c92-432c-b31b-a70650a15e1d tempest-VolumesAdminNegativeTest-1801268990 None] attachment_update completed successfully. Sep 27 10:11:55.292310 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-6a091cb1-4b09-422c-accc-7d57f53c6215 req-6a2be4b2-cd5b-432b-a741-1f390d45bbef tempest-VolumesAdminNegativeTest-1801268990 None] Terminate volume connection completed successfully. Sep 27 10:11:55.293524 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-6a091cb1-4b09-422c-accc-7d57f53c6215 req-6a2be4b2-cd5b-432b-a741-1f390d45bbef tempest-VolumesAdminNegativeTest-1801268990 None] Deleting attachment b58224c9-3fc9-483f-8b43-85dc37cb507f. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:12:06.077562 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=23,cinder:DELETE=2,cinder:UPDATE=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:13.826717 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=44,cinder:UPDATE=12,cinder:INSERT=5,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:13.850584 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-1d9f30b8-3ab4-4264-9275-18982056f191 tempest-TaggedBootDevicesTest-1222598264 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:12:15.016480 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-1d9f30b8-3ab4-4264-9275-18982056f191 tempest-TaggedBootDevicesTest-1222598264 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.166s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:12:15.018324 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-1d9f30b8-3ab4-4264-9275-18982056f191 tempest-TaggedBootDevicesTest-1222598264 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-cceacac7-f31d-4e36-b764-2231c93d64eb', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'cceacac7-f31d-4e36-b764-2231c93d64eb', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:12:15.054797 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-1d9f30b8-3ab4-4264-9275-18982056f191 tempest-TaggedBootDevicesTest-1222598264 None] Connection info returned from driver {'name': 'volumes/volume-cceacac7-f31d-4e36-b764-2231c93d64eb', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'cceacac7-f31d-4e36-b764-2231c93d64eb', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'b96037de-e63f-49eb-92f2-463bd277e3e6'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:12:15.128174 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-1d9f30b8-3ab4-4264-9275-18982056f191 tempest-TaggedBootDevicesTest-1222598264 None] attachment_update completed successfully. Sep 27 10:12:15.785633 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-92292c52-26ad-495f-85e6-2022b3ac1162 tempest-TaggedBootDevicesTest-1222598264 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:12:16.970465 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-92292c52-26ad-495f-85e6-2022b3ac1162 tempest-TaggedBootDevicesTest-1222598264 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.185s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:12:16.971301 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-92292c52-26ad-495f-85e6-2022b3ac1162 tempest-TaggedBootDevicesTest-1222598264 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-8943dda0-13e7-42e3-bdeb-f9cf11b10058', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '8943dda0-13e7-42e3-bdeb-f9cf11b10058', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:12:17.018329 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-92292c52-26ad-495f-85e6-2022b3ac1162 tempest-TaggedBootDevicesTest-1222598264 None] Connection info returned from driver {'name': 'volumes/volume-8943dda0-13e7-42e3-bdeb-f9cf11b10058', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '8943dda0-13e7-42e3-bdeb-f9cf11b10058', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'cf48e7d0-ac5b-44ad-ab11-08b4099b156c'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:12:17.130363 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-92292c52-26ad-495f-85e6-2022b3ac1162 tempest-TaggedBootDevicesTest-1222598264 None] attachment_update completed successfully. Sep 27 10:12:17.532980 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:12:17.593458 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Flow 'volume_create_manager' (e3b7055c-b6d4-403e-9eb0-492c1dd31dd0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:12:17.611896 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (906cf65c-4908-494a-9711-0fffe0cccd88) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:17.635666 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (906cf65c-4908-494a-9711-0fffe0cccd88) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:12:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-1404509110',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a57cf025-29a4-477b-9b8a-c89e9db47889,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f847bee14ff74dfbafe7c6a199f1b99f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:12:17Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:12:17Z,use_quota=True,user_id='b8d6f6c63fe144959139278ca287e2ec',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:17.641395 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (69ee3b85-0aec-4fe6-a9ab-55ea81d3c3f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:17.646136 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (69ee3b85-0aec-4fe6-a9ab-55ea81d3c3f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:17.651338 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (88c56375-69b4-4788-b237-54425e2db5a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:17.655954 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (88c56375-69b4-4788-b237-54425e2db5a3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a57cf025-29a4-477b-9b8a-c89e9db47889', 'volume_name': 'volume-a57cf025-29a4-477b-9b8a-c89e9db47889', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:17.660962 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dce45abb-3b4d-48bb-8740-145584571427) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:17.666206 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dce45abb-3b4d-48bb-8740-145584571427) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:17.672628 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8f504cc8-2a0f-44bf-8247-fa341cf345ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:17.673934 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Volume a57cf025-29a4-477b-9b8a-c89e9db47889: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a57cf025-29a4-477b-9b8a-c89e9db47889', 'volume_size': 1} Sep 27 10:12:17.674585 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] creating volume 'volume-a57cf025-29a4-477b-9b8a-c89e9db47889' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:12:17.675563 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:17.758395 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-84ddd2f1-dea5-47e6-940c-3038c8ffa24e tempest-TaggedBootDevicesTest-1222598264 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:12:17.802152 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:12:17.802705 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:12:17.809569 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8f504cc8-2a0f-44bf-8247-fa341cf345ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a57cf025-29a4-477b-9b8a-c89e9db47889', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:17.824573 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0214b28f-f514-4625-a08d-aa7a3489ca8b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:17.856247 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Volume volume-a57cf025-29a4-477b-9b8a-c89e9db47889 (a57cf025-29a4-477b-9b8a-c89e9db47889): created successfully Sep 27 10:12:17.860997 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0214b28f-f514-4625-a08d-aa7a3489ca8b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:17.867877 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Flow 'volume_create_manager' (e3b7055c-b6d4-403e-9eb0-492c1dd31dd0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:12:17.883083 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-25b22637-c0d8-491e-a59c-3ac644c171e3 tempest-VolumesAdminNegativeTest-1801268990 None] Created volume successfully. Sep 27 10:12:18.823733 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-84ddd2f1-dea5-47e6-940c-3038c8ffa24e tempest-TaggedBootDevicesTest-1222598264 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.066s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:12:18.824546 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-84ddd2f1-dea5-47e6-940c-3038c8ffa24e tempest-TaggedBootDevicesTest-1222598264 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-4b2f9c8c-667b-4bff-8590-c93510f8acf3', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '4b2f9c8c-667b-4bff-8590-c93510f8acf3', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:12:18.866884 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-84ddd2f1-dea5-47e6-940c-3038c8ffa24e tempest-TaggedBootDevicesTest-1222598264 None] Connection info returned from driver {'name': 'volumes/volume-4b2f9c8c-667b-4bff-8590-c93510f8acf3', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '4b2f9c8c-667b-4bff-8590-c93510f8acf3', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'd2678885-9f57-4e5f-b61f-1b2df0c41832'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:12:18.991550 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-3aa31067-4bde-422c-93fb-6b3bf7180858 req-84ddd2f1-dea5-47e6-940c-3038c8ffa24e tempest-TaggedBootDevicesTest-1222598264 None] attachment_update completed successfully. Sep 27 10:12:19.310444 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] Acquiring lock "cinder-a57cf025-29a4-477b-9b8a-c89e9db47889-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:12:19.354666 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] Lock "cinder-a57cf025-29a4-477b-9b8a-c89e9db47889-delete_volume" acquired by "delete_volume" :: waited 0.044s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:12:19.451368 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:19.577154 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:12:19.577604 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] Volume volume-a57cf025-29a4-477b-9b8a-c89e9db47889 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:12:19.600892 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] deleting rbd volume volume-a57cf025-29a4-477b-9b8a-c89e9db47889 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:12:20.072484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] Created reservations ['3541e4fc-40dc-427e-a754-9b47d3929cc2', '551193af-e87d-4d60-b9b0-1e8d981dab8b', 'c4f377a5-9252-41dc-b391-a5c8b2c1c8cb', 'f399cb9e-a2cb-4efd-a38d-1ff741fa383f'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:12:20.170848 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:20.227979 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:20.306144 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:12:20.311243 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] Deleted volume successfully. Sep 27 10:12:20.329962 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-37633607-d51b-49de-b172-f408e4a02977 tempest-VolumesAdminNegativeTest-1801268990 None] Lock "cinder-a57cf025-29a4-477b-9b8a-c89e9db47889-delete_volume" released by "delete_volume" :: held 0.976s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:12:26.128825 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] Acquiring lock "cinder-f605c0e1-aee3-46fe-a624-b9b13bb6cb46-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:12:26.138539 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] Lock "cinder-f605c0e1-aee3-46fe-a624-b9b13bb6cb46-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:12:26.192309 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:26.313068 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:12:26.314634 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] Volume volume-f605c0e1-aee3-46fe-a624-b9b13bb6cb46 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:12:26.343075 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] deleting rbd volume volume-f605c0e1-aee3-46fe-a624-b9b13bb6cb46 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:12:27.205734 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] Created reservations ['7b3fae44-2a50-4a05-b5f1-5e6b7ef76131', '7f05ff1e-386a-4f01-85c5-eea467dd696e', 'df19960e-7fa6-45d3-a302-86f4e1b498ad', '1e953e75-8ad7-4f42-a434-55499a01473e'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:12:27.339306 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:27.438634 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:27.506842 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:12:27.512600 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] Deleted volume successfully. Sep 27 10:12:27.535960 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7cd393f7-23e3-4477-be94-9a4d5164ff46 tempest-VolumesAdminNegativeTest-1801268990 None] Lock "cinder-f605c0e1-aee3-46fe-a624-b9b13bb6cb46-delete_volume" released by "delete_volume" :: held 1.397s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:12:29.728392 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:12:29.731209 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:29.798341 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:29.887683 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:12:37.550308 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6eec02e4-8842-45b5-9c0c-1b07157eff3b tempest-GroupSnapshotsTest-166215701 None] Group tempest-GroupSnapshotsTest-Group-1428896821: creating Sep 27 10:12:37.567039 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=63,cinder:UPDATE=23,cinder:DELETE=1,cinder:INSERT=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:37.635946 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=70,cinder:UPDATE=25,cinder:INSERT=3,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:37.638677 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6eec02e4-8842-45b5-9c0c-1b07157eff3b tempest-GroupSnapshotsTest-166215701 None] Group tempest-GroupSnapshotsTest-Group-1428896821: created successfully Sep 27 10:12:37.660936 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6eec02e4-8842-45b5-9c0c-1b07157eff3b tempest-GroupSnapshotsTest-166215701 None] Create group completed successfully. Sep 27 10:12:39.342691 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:12:39.395748 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Flow 'volume_create_manager' (8bbcf5a6-653e-4a9a-aae6-f793b6b08bfe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:12:39.408580 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5a704f7e-419b-4cb3-909b-88b144297047) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:39.461390 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5a704f7e-419b-4cb3-909b-88b144297047) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:12:39Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-794087237',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=dd1abbe2-14c4-45f9-9a38-4dfab05a1cdd,host='ncdv-master-1@ceph#ceph',id=64aaa7b2-07ce-4eeb-aa0c-1836999ac105,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8734d2a922bc49999e789453be69a0e3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:12:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:12:39Z,use_quota=True,user_id='4ec825af35b24524baee61763cbd431a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(d6759475-f6e3-44f7-aa24-5bab9eec78ce),volume_type_id=d6759475-f6e3-44f7-aa24-5bab9eec78ce)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:39.467668 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3f7ff72a-391c-476e-a75c-08d7b58caa27) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:39.472724 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3f7ff72a-391c-476e-a75c-08d7b58caa27) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:39.482532 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (08fe19e7-3bb8-4875-8358-e7dad485f4a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:39.492100 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (08fe19e7-3bb8-4875-8358-e7dad485f4a3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '64aaa7b2-07ce-4eeb-aa0c-1836999ac105', 'volume_name': 'volume-64aaa7b2-07ce-4eeb-aa0c-1836999ac105', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:39.497500 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (54bbcd01-76a7-415d-a4f5-78878ef5d60d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:39.502287 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (54bbcd01-76a7-415d-a4f5-78878ef5d60d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:39.509326 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d0b7365c-8788-4029-857d-0acf7e25d647) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:39.515228 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Volume 64aaa7b2-07ce-4eeb-aa0c-1836999ac105: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-64aaa7b2-07ce-4eeb-aa0c-1836999ac105', 'volume_size': 1} Sep 27 10:12:39.515630 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] creating volume 'volume-64aaa7b2-07ce-4eeb-aa0c-1836999ac105' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:12:39.516446 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:39.640269 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:12:39.640903 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:12:39.645384 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d0b7365c-8788-4029-857d-0acf7e25d647) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-64aaa7b2-07ce-4eeb-aa0c-1836999ac105', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:39.651870 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c76350de-d6d9-4532-aa4a-8c253b510776) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:39.671180 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Volume volume-64aaa7b2-07ce-4eeb-aa0c-1836999ac105 (64aaa7b2-07ce-4eeb-aa0c-1836999ac105): created successfully Sep 27 10:12:39.676224 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c76350de-d6d9-4532-aa4a-8c253b510776) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:39.685077 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Flow 'volume_create_manager' (8bbcf5a6-653e-4a9a-aae6-f793b6b08bfe) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:12:39.700067 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-73000d50-7353-4256-9470-dd0fd1a7bde0 tempest-GroupSnapshotsTest-166215701 None] Created volume successfully. Sep 27 10:12:41.208199 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-113bd670-7510-46c5-8d8c-c79a0a2fe136 tempest-GroupSnapshotsTest-166215701 None] GroupSnapshot 92064e34-d942-4a44-8c4e-9c4abf331a15: creating. Sep 27 10:12:41.300169 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-113bd670-7510-46c5-8d8c-c79a0a2fe136 tempest-GroupSnapshotsTest-166215701 None] Group snapshot 92064e34-d942-4a44-8c4e-9c4abf331a15: creating. {{(pid=138473) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Sep 27 10:12:41.330547 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-113bd670-7510-46c5-8d8c-c79a0a2fe136 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:42.466962 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-113bd670-7510-46c5-8d8c-c79a0a2fe136 tempest-GroupSnapshotsTest-166215701 None] group_snapshot 92064e34-d942-4a44-8c4e-9c4abf331a15: created successfully Sep 27 10:12:44.377425 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd54b38c-7c58-4d94-bab5-c994a0a1b575 tempest-GroupSnapshotsTest-166215701 None] cloning volumes/volume-64aaa7b2-07ce-4eeb-aa0c-1836999ac105@snapshot-2088004d-e32a-4178-b44a-064824a72e20 to volume-dc2cb09a-90fd-4449-a0a1-c0471a266dd8 {{(pid=138473) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 27 10:12:44.378080 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd54b38c-7c58-4d94-bab5-c994a0a1b575 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:44.466754 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd54b38c-7c58-4d94-bab5-c994a0a1b575 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:44.742789 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd54b38c-7c58-4d94-bab5-c994a0a1b575 tempest-GroupSnapshotsTest-166215701 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:12:44.743832 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd54b38c-7c58-4d94-bab5-c994a0a1b575 tempest-GroupSnapshotsTest-166215701 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:12:44.759364 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd54b38c-7c58-4d94-bab5-c994a0a1b575 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:44.836512 ncdv-master-1 cinder-volume[138473]: WARNING cinder.volume.drivers.rbd [None req-dd54b38c-7c58-4d94-bab5-c994a0a1b575 tempest-GroupSnapshotsTest-166215701 None] Not using v2 clone API, please upgrade to mimic+ and set the OSD minimum client compat version to mimic for better performance, fewer deletion issues: NotImplementedError Sep 27 10:12:45.014844 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-dd54b38c-7c58-4d94-bab5-c994a0a1b575 tempest-GroupSnapshotsTest-166215701 None] Create group from source-snapshot-92064e34-d942-4a44-8c4e-9c4abf331a15 completed successfully. Sep 27 10:12:46.648582 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fc76ac0a-df10-4178-83b9-e6c7bb03cc0b tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:46.780315 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fc76ac0a-df10-4178-83b9-e6c7bb03cc0b tempest-GroupSnapshotsTest-166215701 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:12:46.795152 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fc76ac0a-df10-4178-83b9-e6c7bb03cc0b tempest-GroupSnapshotsTest-166215701 None] deleting rbd volume volume-dc2cb09a-90fd-4449-a0a1-c0471a266dd8 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:12:47.650123 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-fc76ac0a-df10-4178-83b9-e6c7bb03cc0b tempest-GroupSnapshotsTest-166215701 None] Created reservations ['1297af79-8ced-41bc-bd17-9930ef067078'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:12:47.692687 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-fc76ac0a-df10-4178-83b9-e6c7bb03cc0b tempest-GroupSnapshotsTest-166215701 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-1077446857 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-1077446857, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:12:47.693832 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-fc76ac0a-df10-4178-83b9-e6c7bb03cc0b tempest-GroupSnapshotsTest-166215701 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-1077446857 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-1077446857, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:12:47.728987 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-fc76ac0a-df10-4178-83b9-e6c7bb03cc0b tempest-GroupSnapshotsTest-166215701 None] Created reservations ['92abf49c-bbc6-4e1e-b2cf-faeb7e28e63f', '8afd6718-1e2d-4f5c-b7b3-079688d68e06', 'fe856022-44a5-4f00-8df9-d9ba7c7d5fe4', 'f9db0e13-87b4-4eb8-8204-de9dda731de1'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:12:47.915074 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fc76ac0a-df10-4178-83b9-e6c7bb03cc0b tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:48.000576 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fc76ac0a-df10-4178-83b9-e6c7bb03cc0b tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:48.100851 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-fc76ac0a-df10-4178-83b9-e6c7bb03cc0b tempest-GroupSnapshotsTest-166215701 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:12:48.105496 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-fc76ac0a-df10-4178-83b9-e6c7bb03cc0b tempest-GroupSnapshotsTest-166215701 None] Delete group completed successfully. Sep 27 10:12:48.192018 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-20ace5ac-9993-4ba2-9a6e-432a149765a6 tempest-GroupSnapshotsTest-166215701 None] group_snapshot 92064e34-d942-4a44-8c4e-9c4abf331a15: deleting Sep 27 10:12:48.216717 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-20ace5ac-9993-4ba2-9a6e-432a149765a6 tempest-GroupSnapshotsTest-166215701 None] group_snapshot 92064e34-d942-4a44-8c4e-9c4abf331a15: deleting {{(pid=138473) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Sep 27 10:12:48.243906 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-20ace5ac-9993-4ba2-9a6e-432a149765a6 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:49.572557 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-20ace5ac-9993-4ba2-9a6e-432a149765a6 tempest-GroupSnapshotsTest-166215701 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-1077446857 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-1077446857, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:12:49.604911 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-20ace5ac-9993-4ba2-9a6e-432a149765a6 tempest-GroupSnapshotsTest-166215701 None] Created reservations ['bcae58e6-a250-411f-8b5d-0ae80f40599a', '9c5e943e-db21-405d-ac09-d654ce50190c', 'b54ab2ea-c27d-443c-9932-520173a9b78b', '6f2feae1-038d-44fe-8bac-11e809326ed0'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:12:49.701343 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-20ace5ac-9993-4ba2-9a6e-432a149765a6 tempest-GroupSnapshotsTest-166215701 None] group_snapshot 92064e34-d942-4a44-8c4e-9c4abf331a15: deleted successfully Sep 27 10:12:51.012999 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=71,cinder:UPDATE=26,cinder:DELETE=1,cinder:INSERT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:51.027957 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70796e1f-65ea-4ebc-95ca-54e7e6497a79 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:51.156714 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70796e1f-65ea-4ebc-95ca-54e7e6497a79 tempest-GroupSnapshotsTest-166215701 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:12:51.158466 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70796e1f-65ea-4ebc-95ca-54e7e6497a79 tempest-GroupSnapshotsTest-166215701 None] Volume volume-64aaa7b2-07ce-4eeb-aa0c-1836999ac105 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:12:51.174839 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70796e1f-65ea-4ebc-95ca-54e7e6497a79 tempest-GroupSnapshotsTest-166215701 None] deleting rbd volume volume-64aaa7b2-07ce-4eeb-aa0c-1836999ac105 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:12:52.106525 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-70796e1f-65ea-4ebc-95ca-54e7e6497a79 tempest-GroupSnapshotsTest-166215701 None] Created reservations ['2265e0dc-c269-4d14-93e6-c451e0d529ee'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:12:52.182579 ncdv-master-1 cinder-volume[138473]: WARNING cinder.db.sqlalchemy.api [None req-70796e1f-65ea-4ebc-95ca-54e7e6497a79 tempest-GroupSnapshotsTest-166215701 None] Reservation would make usage less than 0 for the following resources, so on commit they will be limited to prevent going below 0: ['gigabytes', 'gigabytes_tempest-GroupSnapshotsTest-volume-type-1077446857'] Sep 27 10:12:52.191019 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-70796e1f-65ea-4ebc-95ca-54e7e6497a79 tempest-GroupSnapshotsTest-166215701 None] Created reservations ['c6ee5ccd-4ab0-43b8-a5b5-ab9c67335833', '00c6e9fa-03ed-49d6-9757-1aa7864db305', '76484da8-1ca5-4bf2-ad61-c9cdb43cacd5', '8e01852f-503c-41c4-bb25-bdded5268e26'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:12:52.276486 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=24,cinder:SELECT=68,cinder:INSERT=8 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:52.408958 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70796e1f-65ea-4ebc-95ca-54e7e6497a79 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:52.473722 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70796e1f-65ea-4ebc-95ca-54e7e6497a79 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:52.540702 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-70796e1f-65ea-4ebc-95ca-54e7e6497a79 tempest-GroupSnapshotsTest-166215701 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:12:52.559644 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-70796e1f-65ea-4ebc-95ca-54e7e6497a79 tempest-GroupSnapshotsTest-166215701 None] Delete group completed successfully. Sep 27 10:12:53.877030 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-9d18f199-abff-431a-b6a9-33877a211e04 tempest-GroupSnapshotsTest-166215701 None] Group tempest-GroupSnapshotsTest-Group-1647746219: creating Sep 27 10:12:53.927554 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-9d18f199-abff-431a-b6a9-33877a211e04 tempest-GroupSnapshotsTest-166215701 None] Group tempest-GroupSnapshotsTest-Group-1647746219: created successfully Sep 27 10:12:53.950555 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-9d18f199-abff-431a-b6a9-33877a211e04 tempest-GroupSnapshotsTest-166215701 None] Create group completed successfully. Sep 27 10:12:55.449030 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:12:55.489191 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Flow 'volume_create_manager' (5a95dce9-ad3a-4f5c-83a3-c97b41f1c514) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:12:55.496773 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (77fda006-8781-4382-9b0b-267e72495821) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:55.515513 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (77fda006-8781-4382-9b0b-267e72495821) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:12:55Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1006821907',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=80e3d5d6-3c0c-4630-9b6d-5207207dd359,host='ncdv-master-1@ceph#ceph',id=4cf8e156-327c-46c5-a639-1ef2ee984fc3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8734d2a922bc49999e789453be69a0e3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:12:55Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:12:55Z,use_quota=True,user_id='4ec825af35b24524baee61763cbd431a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(3b7382a0-131a-4559-a5ac-4d4d951f3d81),volume_type_id=3b7382a0-131a-4559-a5ac-4d4d951f3d81)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:55.522457 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3bd051ae-af11-49ef-9525-3ebf331c7c66) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:55.527491 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3bd051ae-af11-49ef-9525-3ebf331c7c66) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:55.534187 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e3808463-fe29-4cbe-abf0-81d0f89f8094) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:55.539944 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e3808463-fe29-4cbe-abf0-81d0f89f8094) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4cf8e156-327c-46c5-a639-1ef2ee984fc3', 'volume_name': 'volume-4cf8e156-327c-46c5-a639-1ef2ee984fc3', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:55.545257 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c8c475b6-9880-4844-8e47-7801ce615a31) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:55.549308 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c8c475b6-9880-4844-8e47-7801ce615a31) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:55.554419 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (76e15502-c993-4278-9d79-8a15efdf0925) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:55.555557 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Volume 4cf8e156-327c-46c5-a639-1ef2ee984fc3: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4cf8e156-327c-46c5-a639-1ef2ee984fc3', 'volume_size': 1} Sep 27 10:12:55.556114 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] creating volume 'volume-4cf8e156-327c-46c5-a639-1ef2ee984fc3' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:12:55.557065 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:55.730724 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:12:55.731103 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:12:55.735110 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (76e15502-c993-4278-9d79-8a15efdf0925) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4cf8e156-327c-46c5-a639-1ef2ee984fc3', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:55.744814 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (08485258-108e-402a-9ef4-2073cae47602) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:55.758039 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Volume volume-4cf8e156-327c-46c5-a639-1ef2ee984fc3 (4cf8e156-327c-46c5-a639-1ef2ee984fc3): created successfully Sep 27 10:12:55.762404 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (08485258-108e-402a-9ef4-2073cae47602) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:55.768220 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Flow 'volume_create_manager' (5a95dce9-ad3a-4f5c-83a3-c97b41f1c514) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:12:55.779980 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-9f08c29a-02fe-4317-9d9b-ca3d27175720 tempest-GroupSnapshotsTest-166215701 None] Created volume successfully. Sep 27 10:12:57.251680 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:12:57.299577 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Flow 'volume_create_manager' (d1a4e064-c229-4b0a-9eee-5d65669baa8a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:12:57.333120 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c47531a3-6e1c-4cb4-b85d-836362851150) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:57.384367 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c47531a3-6e1c-4cb4-b85d-836362851150) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:12:57Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1267369827',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=80e3d5d6-3c0c-4630-9b6d-5207207dd359,host='ncdv-master-1@ceph#ceph',id=55d0f508-a4e0-476d-a57a-45db34e6d8ee,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8734d2a922bc49999e789453be69a0e3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:12:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:12:57Z,use_quota=True,user_id='4ec825af35b24524baee61763cbd431a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(3b7382a0-131a-4559-a5ac-4d4d951f3d81),volume_type_id=3b7382a0-131a-4559-a5ac-4d4d951f3d81)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:57.405888 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e6acc9a4-6bad-4e2e-ad95-a7e748265d76) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:57.417294 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e6acc9a4-6bad-4e2e-ad95-a7e748265d76) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:57.428399 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (74650f9a-fa64-4078-ba80-e5b3cd7d9af0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:57.440805 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (74650f9a-fa64-4078-ba80-e5b3cd7d9af0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '55d0f508-a4e0-476d-a57a-45db34e6d8ee', 'volume_name': 'volume-55d0f508-a4e0-476d-a57a-45db34e6d8ee', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:57.460402 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (907a9c1f-dc31-4f55-bf61-f51adc961b40) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:57.487066 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (907a9c1f-dc31-4f55-bf61-f51adc961b40) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:57.499972 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d08507e0-dfeb-47a5-b3c6-8b093944eb45) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:57.501255 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Volume 55d0f508-a4e0-476d-a57a-45db34e6d8ee: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-55d0f508-a4e0-476d-a57a-45db34e6d8ee', 'volume_size': 1} Sep 27 10:12:57.501649 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] creating volume 'volume-55d0f508-a4e0-476d-a57a-45db34e6d8ee' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:12:57.502367 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:12:57.648847 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:12:57.649640 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:12:57.653859 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d08507e0-dfeb-47a5-b3c6-8b093944eb45) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-55d0f508-a4e0-476d-a57a-45db34e6d8ee', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:57.677881 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a4d7da08-3064-4c00-ae1a-0f5183d91463) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:12:57.702321 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Volume volume-55d0f508-a4e0-476d-a57a-45db34e6d8ee (55d0f508-a4e0-476d-a57a-45db34e6d8ee): created successfully Sep 27 10:12:57.708858 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a4d7da08-3064-4c00-ae1a-0f5183d91463) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:12:57.722161 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Flow 'volume_create_manager' (d1a4e064-c229-4b0a-9eee-5d65669baa8a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:12:57.746108 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-98c0c5aa-38f6-4117-8725-80a663943440 tempest-GroupSnapshotsTest-166215701 None] Created volume successfully. Sep 27 10:12:58.579425 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d0e3f253-7d44-48e6-9bc5-7254babd1472 tempest-GroupSnapshotsTest-166215701 None] Update group completed successfully. Sep 27 10:12:59.819755 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-63600167-b952-4e67-9ce1-00768a0cb74e tempest-GroupSnapshotsTest-166215701 None] Update group completed successfully. Sep 27 10:13:01.022224 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1807ee5d-d474-4cf1-bb15-b8d5e290d3ed tempest-GroupSnapshotsTest-166215701 None] Update group completed successfully. Sep 27 10:13:02.303416 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-23fc94c1-1a67-4c85-ad1f-47c46ab49e06 tempest-GroupSnapshotsTest-166215701 None] Update group completed successfully. Sep 27 10:13:03.898929 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-563aa4d4-78cb-4fb3-a2b5-4e3399b08afd tempest-GroupSnapshotsTest-166215701 None] GroupSnapshot c109d143-386b-4493-8a44-12fe3dcd529f: creating. Sep 27 10:13:03.941544 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-563aa4d4-78cb-4fb3-a2b5-4e3399b08afd tempest-GroupSnapshotsTest-166215701 None] Group snapshot c109d143-386b-4493-8a44-12fe3dcd529f: creating. {{(pid=138473) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Sep 27 10:13:03.966123 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-563aa4d4-78cb-4fb3-a2b5-4e3399b08afd tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:05.301211 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-563aa4d4-78cb-4fb3-a2b5-4e3399b08afd tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:06.589789 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=70,cinder:UPDATE=25,cinder:INSERT=4,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:06.598143 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-563aa4d4-78cb-4fb3-a2b5-4e3399b08afd tempest-GroupSnapshotsTest-166215701 None] group_snapshot c109d143-386b-4493-8a44-12fe3dcd529f: created successfully Sep 27 10:13:07.465853 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-be7788de-848c-462a-801d-f4a70557c013 tempest-GroupSnapshotsTest-166215701 None] group_snapshot c109d143-386b-4493-8a44-12fe3dcd529f: deleting Sep 27 10:13:07.518163 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-be7788de-848c-462a-801d-f4a70557c013 tempest-GroupSnapshotsTest-166215701 None] group_snapshot c109d143-386b-4493-8a44-12fe3dcd529f: deleting {{(pid=138473) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Sep 27 10:13:07.574595 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-be7788de-848c-462a-801d-f4a70557c013 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:08.407286 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-be7788de-848c-462a-801d-f4a70557c013 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:09.440987 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-be7788de-848c-462a-801d-f4a70557c013 tempest-GroupSnapshotsTest-166215701 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-1862578642 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-1862578642, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:13:09.442666 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-be7788de-848c-462a-801d-f4a70557c013 tempest-GroupSnapshotsTest-166215701 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-1862578642 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-1862578642, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:13:09.463835 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-be7788de-848c-462a-801d-f4a70557c013 tempest-GroupSnapshotsTest-166215701 None] Created reservations ['2526790f-d15f-438d-b186-1d4c98cbfdf3', 'd480d92a-0049-44ec-82a8-372945a806fc', 'e525dda9-3835-4d2d-acf7-8fc5b9c6426d', '43f946f1-f16c-4671-aed2-dcc16e4ab791'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:13:09.624120 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=72,cinder:UPDATE=22,cinder:DELETE=1,cinder:INSERT=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:09.633314 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-be7788de-848c-462a-801d-f4a70557c013 tempest-GroupSnapshotsTest-166215701 None] Created reservations ['5537ce61-8557-4ed9-b1bc-777d402f3684', '7bfbe6be-72f0-4645-b693-9f220847d21a', '9e3205ec-b376-4835-9432-0ebb932fae99', 'e9f4079b-1584-490b-b650-5cb91dd8211e'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:13:09.742495 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-be7788de-848c-462a-801d-f4a70557c013 tempest-GroupSnapshotsTest-166215701 None] group_snapshot c109d143-386b-4493-8a44-12fe3dcd529f: deleted successfully Sep 27 10:13:10.544096 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:10.668154 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:13:10.668880 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] Volume volume-4cf8e156-327c-46c5-a639-1ef2ee984fc3 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:13:10.683699 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] deleting rbd volume volume-4cf8e156-327c-46c5-a639-1ef2ee984fc3 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:13:11.123658 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:11.219987 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:13:11.220569 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] Volume volume-55d0f508-a4e0-476d-a57a-45db34e6d8ee is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:13:11.237469 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] deleting rbd volume volume-55d0f508-a4e0-476d-a57a-45db34e6d8ee {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:13:11.872500 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] Created reservations ['b5cdd560-f5b9-47a6-8879-23d408ce77ce'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:13:11.925732 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-1862578642 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-1862578642, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:13:11.954142 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] Created reservations ['ba33fa6d-4cca-4589-8c36-7de968116480', '6dcdf5c5-fe62-4487-b929-1efcb984cfca', 'b5b979b3-2518-4b8c-8c78-68a462e9dce3', '987c2b7a-545f-4063-9552-826efc78b357'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:13:12.118626 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] Created reservations ['daec1754-1a63-4a3a-ac1a-119e086ac2ef', 'dc1363d9-55cd-468c-a5f5-62f01d424464', '4e0a603d-19de-4ccd-b14b-ecebd4a802c8', '270d3939-ed42-495a-80b1-f188388e4c14'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:13:12.297282 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:12.394731 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:12.457798 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:13:12.464152 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-41523212-32f8-49a0-b644-b7534f879206 tempest-GroupSnapshotsTest-166215701 None] Delete group completed successfully. Sep 27 10:13:13.404689 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=54,cinder:UPDATE=37,cinder:INSERT=9 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:13.423117 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-598ac7aa-4266-438f-8231-5265531d7e7a tempest-GroupSnapshotsTest-166215701 None] Group tempest-GroupSnapshotsTest-Group-885710130: creating Sep 27 10:13:13.485099 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-598ac7aa-4266-438f-8231-5265531d7e7a tempest-GroupSnapshotsTest-166215701 None] Group tempest-GroupSnapshotsTest-Group-885710130: created successfully Sep 27 10:13:13.511699 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-598ac7aa-4266-438f-8231-5265531d7e7a tempest-GroupSnapshotsTest-166215701 None] Create group completed successfully. Sep 27 10:13:15.506537 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:13:15.556480 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Flow 'volume_create_manager' (00b68b34-b036-49bd-8040-139edb82bbcb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:13:15.565070 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (03a562d4-fa9a-4ece-afa0-338e574c7fd9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:15.588318 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (03a562d4-fa9a-4ece-afa0-338e574c7fd9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:13:15Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1636092091',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=a4049ba1-e34d-4cdd-80c6-9120b4f3fbde,host='ncdv-master-1@ceph#ceph',id=37826c9e-c8ab-4d7a-9b12-2cc0bd202d4f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8734d2a922bc49999e789453be69a0e3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:13:15Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:13:15Z,use_quota=True,user_id='4ec825af35b24524baee61763cbd431a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e1ba8c64-aaaf-4d20-b781-e09381e5e2af),volume_type_id=e1ba8c64-aaaf-4d20-b781-e09381e5e2af)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:15.619918 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (32cb2c9d-5bba-4540-a2f7-2f44c7beb557) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:15.632918 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (32cb2c9d-5bba-4540-a2f7-2f44c7beb557) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:15.643246 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ce27a604-0cea-4da1-8a40-167eb106d6de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:15.657565 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ce27a604-0cea-4da1-8a40-167eb106d6de) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '37826c9e-c8ab-4d7a-9b12-2cc0bd202d4f', 'volume_name': 'volume-37826c9e-c8ab-4d7a-9b12-2cc0bd202d4f', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:15.664337 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3c26613b-8359-4419-a585-95f483522771) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:15.669491 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3c26613b-8359-4419-a585-95f483522771) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:15.680253 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0842d396-f535-47b2-92c4-989e22b83369) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:15.681702 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Volume 37826c9e-c8ab-4d7a-9b12-2cc0bd202d4f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-37826c9e-c8ab-4d7a-9b12-2cc0bd202d4f', 'volume_size': 1} Sep 27 10:13:15.682476 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] creating volume 'volume-37826c9e-c8ab-4d7a-9b12-2cc0bd202d4f' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:13:15.683416 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:15.891593 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:13:15.892106 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:13:15.896122 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0842d396-f535-47b2-92c4-989e22b83369) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-37826c9e-c8ab-4d7a-9b12-2cc0bd202d4f', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:15.901431 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cd4870b1-1b93-455a-be2e-87312663e75f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:15.920864 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Volume volume-37826c9e-c8ab-4d7a-9b12-2cc0bd202d4f (37826c9e-c8ab-4d7a-9b12-2cc0bd202d4f): created successfully Sep 27 10:13:15.926641 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cd4870b1-1b93-455a-be2e-87312663e75f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:15.938574 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Flow 'volume_create_manager' (00b68b34-b036-49bd-8040-139edb82bbcb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:13:15.973486 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b45a4dd2-b3ea-4c76-91d3-cfe90995cee4 tempest-GroupSnapshotsTest-166215701 None] Created volume successfully. Sep 27 10:13:16.946389 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1ee522dd-fb4b-4828-a680-bd5a836e0727 tempest-GroupSnapshotsTest-166215701 None] GroupSnapshot e7fe1989-1dd0-419b-bdba-c85dafe09d0d: creating. Sep 27 10:13:16.983043 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1ee522dd-fb4b-4828-a680-bd5a836e0727 tempest-GroupSnapshotsTest-166215701 None] Group snapshot e7fe1989-1dd0-419b-bdba-c85dafe09d0d: creating. {{(pid=138473) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Sep 27 10:13:17.030229 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1ee522dd-fb4b-4828-a680-bd5a836e0727 tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:18.289942 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1ee522dd-fb4b-4828-a680-bd5a836e0727 tempest-GroupSnapshotsTest-166215701 None] group_snapshot e7fe1989-1dd0-419b-bdba-c85dafe09d0d: created successfully Sep 27 10:13:19.489940 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0c51bb3a-9fc8-4875-b520-8e6e68f4984e tempest-GroupSnapshotsTest-166215701 None] group_snapshot e7fe1989-1dd0-419b-bdba-c85dafe09d0d: deleting Sep 27 10:13:19.527103 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0c51bb3a-9fc8-4875-b520-8e6e68f4984e tempest-GroupSnapshotsTest-166215701 None] group_snapshot e7fe1989-1dd0-419b-bdba-c85dafe09d0d: deleting {{(pid=138473) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Sep 27 10:13:19.576916 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0c51bb3a-9fc8-4875-b520-8e6e68f4984e tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:20.321393 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-0c51bb3a-9fc8-4875-b520-8e6e68f4984e tempest-GroupSnapshotsTest-166215701 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-2095435673 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-2095435673, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:13:20.322462 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-0c51bb3a-9fc8-4875-b520-8e6e68f4984e tempest-GroupSnapshotsTest-166215701 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-2095435673 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-2095435673, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:13:20.361660 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-0c51bb3a-9fc8-4875-b520-8e6e68f4984e tempest-GroupSnapshotsTest-166215701 None] Created reservations ['44e3962a-f150-43c1-b2af-3779945569a6', '6b750314-f947-425d-8407-ca677091b428', 'd8e68dd7-3aa2-476e-bca2-368828e8709f', 'ab834e81-7ae8-4d95-87e5-7a270f57ca23'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:13:20.448530 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0c51bb3a-9fc8-4875-b520-8e6e68f4984e tempest-GroupSnapshotsTest-166215701 None] group_snapshot e7fe1989-1dd0-419b-bdba-c85dafe09d0d: deleted successfully Sep 27 10:13:21.434063 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6f2a1a8b-d31d-49a6-a6a1-fd9ee224f58a tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:21.570651 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6f2a1a8b-d31d-49a6-a6a1-fd9ee224f58a tempest-GroupSnapshotsTest-166215701 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:13:21.571513 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6f2a1a8b-d31d-49a6-a6a1-fd9ee224f58a tempest-GroupSnapshotsTest-166215701 None] Volume volume-37826c9e-c8ab-4d7a-9b12-2cc0bd202d4f is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:13:21.588535 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6f2a1a8b-d31d-49a6-a6a1-fd9ee224f58a tempest-GroupSnapshotsTest-166215701 None] deleting rbd volume volume-37826c9e-c8ab-4d7a-9b12-2cc0bd202d4f {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:13:22.054332 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=68,cinder:UPDATE=28,cinder:INSERT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:22.066806 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-6f2a1a8b-d31d-49a6-a6a1-fd9ee224f58a tempest-GroupSnapshotsTest-166215701 None] Created reservations ['eeae32b4-ae82-4fcd-ae80-9225b17165a9'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:13:22.100342 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-6f2a1a8b-d31d-49a6-a6a1-fd9ee224f58a tempest-GroupSnapshotsTest-166215701 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-2095435673 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-2095435673, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:13:22.136219 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-6f2a1a8b-d31d-49a6-a6a1-fd9ee224f58a tempest-GroupSnapshotsTest-166215701 None] Created reservations ['941de77f-14bb-4f10-bbb5-c8ecb9b9a074', '4d37e40c-e6dd-42c9-b0af-5a499bb22efe', '580ad93a-b790-45b7-8b0d-146e77b184b2', 'b1718e27-dd83-4f27-b545-62c467b107b2'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:13:22.296643 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6f2a1a8b-d31d-49a6-a6a1-fd9ee224f58a tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:22.366182 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6f2a1a8b-d31d-49a6-a6a1-fd9ee224f58a tempest-GroupSnapshotsTest-166215701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:22.443671 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-6f2a1a8b-d31d-49a6-a6a1-fd9ee224f58a tempest-GroupSnapshotsTest-166215701 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:13:22.448614 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6f2a1a8b-d31d-49a6-a6a1-fd9ee224f58a tempest-GroupSnapshotsTest-166215701 None] Delete group completed successfully. Sep 27 10:13:29.722659 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:13:29.724472 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:29.816329 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:29.914473 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:13:36.078913 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=58,cinder:UPDATE=19,cinder:DELETE=1,cinder:INSERT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:40.207111 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:13:40.239201 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Flow 'volume_create_manager' (2d9ec078-a0e4-4113-a1f7-c34d6e6d5c93) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:13:40.248522 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c51dbdd0-9492-462d-92d1-21d3e6e90e50) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:40.268723 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c51dbdd0-9492-462d-92d1-21d3e6e90e50) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:13:40Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasNegativeTestJSON-Volume-983333913',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=516a0d90-a10e-4763-9ce9-e1cfc9d99d5f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='18102a849cbe4b16a0d874e0240630de',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:13:40Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:13:40Z,use_quota=True,user_id='c258dd49c93c48aca11be52aadf62ae9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:40.274435 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a416f720-6c1b-4620-8217-68c388af4643) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:40.284466 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a416f720-6c1b-4620-8217-68c388af4643) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:40.291884 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b98c9cf4-7dbc-4129-ba19-6eafb7794254) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:40.299115 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b98c9cf4-7dbc-4129-ba19-6eafb7794254) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '516a0d90-a10e-4763-9ce9-e1cfc9d99d5f', 'volume_name': 'volume-516a0d90-a10e-4763-9ce9-e1cfc9d99d5f', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:40.305962 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fc3efe4a-9e44-47dc-92ba-10f2591b53a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:40.310877 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fc3efe4a-9e44-47dc-92ba-10f2591b53a3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:40.320365 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (803a57d6-631b-4204-8767-984e8003d473) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:40.323825 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Volume 516a0d90-a10e-4763-9ce9-e1cfc9d99d5f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-516a0d90-a10e-4763-9ce9-e1cfc9d99d5f', 'volume_size': 1} Sep 27 10:13:40.324371 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] creating volume 'volume-516a0d90-a10e-4763-9ce9-e1cfc9d99d5f' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:13:40.325362 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:40.471903 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:13:40.472396 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:13:40.479320 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (803a57d6-631b-4204-8767-984e8003d473) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-516a0d90-a10e-4763-9ce9-e1cfc9d99d5f', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:40.485858 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fd74a5df-fe7a-4e61-bb49-732c9fe1d36c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:40.502209 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Volume volume-516a0d90-a10e-4763-9ce9-e1cfc9d99d5f (516a0d90-a10e-4763-9ce9-e1cfc9d99d5f): created successfully Sep 27 10:13:40.508887 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fd74a5df-fe7a-4e61-bb49-732c9fe1d36c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:40.515932 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Flow 'volume_create_manager' (2d9ec078-a0e4-4113-a1f7-c34d6e6d5c93) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:13:40.529602 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-f80e9221-bd30-4377-829b-fe65f9d8189e tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Created volume successfully. Sep 27 10:13:42.414645 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Acquiring lock "cinder-516a0d90-a10e-4763-9ce9-e1cfc9d99d5f-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:13:42.440839 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Lock "cinder-516a0d90-a10e-4763-9ce9-e1cfc9d99d5f-delete_volume" acquired by "delete_volume" :: waited 0.026s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:13:42.492846 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:42.571319 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:13:42.572424 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Volume volume-516a0d90-a10e-4763-9ce9-e1cfc9d99d5f is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:13:42.578761 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] deleting rbd volume volume-516a0d90-a10e-4763-9ce9-e1cfc9d99d5f {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:13:42.837110 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Created reservations ['ab533e90-f662-4b53-b1f9-516ba57cbb7b', 'c564e562-414f-4a5b-baee-2a271d54257e', 'b9a2cfec-d76e-4d0d-bd7f-775c136c8ac5', '5e8d94a2-1a87-4458-9bd2-57c7dfa1de09'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:13:42.902926 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:42.962004 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:43.043572 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:13:43.049184 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Deleted volume successfully. Sep 27 10:13:43.072169 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-85d97697-3a23-4fb2-a59e-a6f5098d01c5 tempest-VolumeQuotasNegativeTestJSON-1125706242 None] Lock "cinder-516a0d90-a10e-4763-9ce9-e1cfc9d99d5f-delete_volume" released by "delete_volume" :: held 0.631s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:13:51.377024 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:13:51.397919 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Flow 'volume_create_manager' (4492fd04-161a-4ae8-a33e-1c8f7856f389) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:13:51.404631 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e85cf864-88fb-47c8-9fd1-2553b5bac9d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:51.420878 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e85cf864-88fb-47c8-9fd1-2553b5bac9d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:13:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesServicesTestJSON-Volume-1047536758',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=08503eaa-2eb4-4257-b1bc-77a3313bf60f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ef3736d32d44426d921789191fdbe507',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:13:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:13:51Z,use_quota=True,user_id='04c6e14a03984c97b9ef7df71dcf4f1d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:51.426182 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (43b90bf4-505d-44ff-b8b1-ab078eeb8458) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:51.430948 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (43b90bf4-505d-44ff-b8b1-ab078eeb8458) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:51.436388 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1016b09c-e878-4eaa-9c4b-0bcdf46e39bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:51.441560 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1016b09c-e878-4eaa-9c4b-0bcdf46e39bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '08503eaa-2eb4-4257-b1bc-77a3313bf60f', 'volume_name': 'volume-08503eaa-2eb4-4257-b1bc-77a3313bf60f', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:51.446390 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (069326f9-3dc7-476d-b8e5-5d041afdfbdb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:51.450972 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (069326f9-3dc7-476d-b8e5-5d041afdfbdb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:51.456055 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (44fc01d6-4ec7-4369-a0f6-e9e852a855d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:51.457421 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Volume 08503eaa-2eb4-4257-b1bc-77a3313bf60f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-08503eaa-2eb4-4257-b1bc-77a3313bf60f', 'volume_size': 1} Sep 27 10:13:51.457954 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] creating volume 'volume-08503eaa-2eb4-4257-b1bc-77a3313bf60f' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:13:51.458876 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:51.540904 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:13:51.541281 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:13:51.546724 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (44fc01d6-4ec7-4369-a0f6-e9e852a855d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-08503eaa-2eb4-4257-b1bc-77a3313bf60f', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:51.551524 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b96e57d6-18f2-407b-bfbc-6a0413f9e55d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:13:51.566897 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Volume volume-08503eaa-2eb4-4257-b1bc-77a3313bf60f (08503eaa-2eb4-4257-b1bc-77a3313bf60f): created successfully Sep 27 10:13:51.573949 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b96e57d6-18f2-407b-bfbc-6a0413f9e55d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:13:51.584162 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Flow 'volume_create_manager' (4492fd04-161a-4ae8-a33e-1c8f7856f389) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:13:51.596405 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-17c8fb47-18c5-40d2-8bac-d237bb11826e tempest-VolumesServicesTestJSON-914391284 None] Created volume successfully. Sep 27 10:13:52.722068 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] Acquiring lock "cinder-08503eaa-2eb4-4257-b1bc-77a3313bf60f-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:13:52.744957 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] Lock "cinder-08503eaa-2eb4-4257-b1bc-77a3313bf60f-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:13:52.752436 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=40,cinder:INSERT=6,cinder:UPDATE=29,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:52.813427 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:52.886021 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:13:52.887001 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] Volume volume-08503eaa-2eb4-4257-b1bc-77a3313bf60f is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:13:52.905744 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] deleting rbd volume volume-08503eaa-2eb4-4257-b1bc-77a3313bf60f {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:13:53.389807 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] Created reservations ['a8945a41-77ee-4d04-9b8f-8bb6aa41325e', '3fef627a-01ab-4412-b61b-975927a0b202', '17010a30-acd7-4418-a185-6962b74262f2', 'e68ed0b2-8835-480f-ae6e-a04123f92069'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:13:53.481375 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:53.545199 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:13:53.583494 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:13:53.588194 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] Deleted volume successfully. Sep 27 10:13:53.617236 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-9043104a-a9ad-4443-be59-a87c3cf735f8 tempest-VolumesServicesTestJSON-914391284 None] Lock "cinder-08503eaa-2eb4-4257-b1bc-77a3313bf60f-delete_volume" released by "delete_volume" :: held 0.884s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:14:02.636791 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:14:02.670013 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Flow 'volume_create_manager' (a8d87202-c965-4056-8cde-f998676f71bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:14:02.679102 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (26c4afd9-c347-4279-a62f-b9157686a37f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:02.715093 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (26c4afd9-c347-4279-a62f-b9157686a37f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:14:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeTypesTest-volume-215761099',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6c82bada-c6f1-425a-88a5-8516c13a36df,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f7e2c5717b264d81900ff088a424fc34',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:14:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:14:03Z,use_quota=True,user_id='de3d102d4ea34b2981829813e211a010',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(1b0afe95-face-4763-bf01-cdd4f062ff92),volume_type_id=1b0afe95-face-4763-bf01-cdd4f062ff92)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:02.722184 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9471a27a-2232-4d81-894c-573b974b05b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:02.729045 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9471a27a-2232-4d81-894c-573b974b05b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:02.738672 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bb193f0e-d75d-40b9-99d5-56eefda438a9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:02.744237 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bb193f0e-d75d-40b9-99d5-56eefda438a9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6c82bada-c6f1-425a-88a5-8516c13a36df', 'volume_name': 'volume-6c82bada-c6f1-425a-88a5-8516c13a36df', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:02.750406 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (80fbb684-3bde-49c0-92dc-f3efb0178bcd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:02.755282 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (80fbb684-3bde-49c0-92dc-f3efb0178bcd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:02.761886 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (85300318-028f-4cd9-bef8-d3217d27f037) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:02.763534 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Volume 6c82bada-c6f1-425a-88a5-8516c13a36df: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6c82bada-c6f1-425a-88a5-8516c13a36df', 'volume_size': 1} Sep 27 10:14:02.764132 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] creating volume 'volume-6c82bada-c6f1-425a-88a5-8516c13a36df' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:14:02.765386 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:02.937887 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] extra_specs: {'storage_protocol': 'ceph', 'vendor_name': 'Open Source'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:14:02.938531 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] extra_specs: {'storage_protocol': 'ceph', 'vendor_name': 'Open Source'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:14:02.943893 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (85300318-028f-4cd9-bef8-d3217d27f037) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6c82bada-c6f1-425a-88a5-8516c13a36df', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:02.949589 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4860f235-219e-462a-86f7-5929fbdb057f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:02.981920 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Volume volume-6c82bada-c6f1-425a-88a5-8516c13a36df (6c82bada-c6f1-425a-88a5-8516c13a36df): created successfully Sep 27 10:14:02.985686 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4860f235-219e-462a-86f7-5929fbdb057f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:02.997147 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Flow 'volume_create_manager' (a8d87202-c965-4056-8cde-f998676f71bb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:14:03.031677 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-328e6090-e49f-444e-bbaa-6b9e7b5a0ef5 tempest-VolumeTypesTest-516719792 None] Created volume successfully. Sep 27 10:14:04.376482 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89c97910-df5e-43c9-b23d-8f8de7981c5d tempest-VolumeTypesTest-516719792 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:04.476179 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-89c97910-df5e-43c9-b23d-8f8de7981c5d tempest-VolumeTypesTest-516719792 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:04.554297 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-89c97910-df5e-43c9-b23d-8f8de7981c5d tempest-VolumeTypesTest-516719792 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:14:04.558373 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-89c97910-df5e-43c9-b23d-8f8de7981c5d tempest-VolumeTypesTest-516719792 None] Retype volume completed successfully. Sep 27 10:14:06.611806 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] Acquiring lock "cinder-6c82bada-c6f1-425a-88a5-8516c13a36df-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:14:06.631881 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] Lock "cinder-6c82bada-c6f1-425a-88a5-8516c13a36df-delete_volume" acquired by "delete_volume" :: waited 0.020s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:14:06.664494 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=53,cinder:UPDATE=26,cinder:INSERT=5,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:06.728622 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:06.837248 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:14:06.837670 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] Volume volume-6c82bada-c6f1-425a-88a5-8516c13a36df is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:14:06.857149 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] deleting rbd volume volume-6c82bada-c6f1-425a-88a5-8516c13a36df {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:14:07.533137 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-1662255683 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-1662255683, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:14:07.533944 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-1662255683 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-1662255683, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:14:07.557086 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] Created reservations ['86d251ff-113a-4fac-ae39-77a67cc6dcf7', 'dd7a2abb-ee53-4d08-92e3-19d4c34b87aa', 'c2790447-a1eb-49c9-8efa-01bda0fb96c5', '0dc2313b-cb77-4737-8832-cbbe7564ce08'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:14:07.662561 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:07.800485 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:07.899412 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:14:07.903908 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] Deleted volume successfully. Sep 27 10:14:07.928643 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-949a539c-4f1c-46f4-820a-f07265f96752 tempest-VolumeTypesTest-516719792 None] Lock "cinder-6c82bada-c6f1-425a-88a5-8516c13a36df-delete_volume" released by "delete_volume" :: held 1.297s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:14:24.748735 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=45,cinder:INSERT=4,cinder:UPDATE=22,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:24.766521 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:14:24.802489 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Flow 'volume_create_manager' (4d7c0de8-16f6-4e23-bdb9-586bdee4820e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:14:24.813622 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (179a5dc8-e621-4457-bb08-c8335df0f391) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:24.838139 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (179a5dc8-e621-4457-bb08-c8335df0f391) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:14:24Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-1777898769',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e5af107c-5b71-461c-a3e1-deb10425a0f9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e2e00c90216b4d1b80ac7892ee23550f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:14:25Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:14:25Z,use_quota=True,user_id='279d3662f60c4a979c22ee3e8fa92865',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:24.846326 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7556e27b-cf84-48f1-8c80-b30ba33c94ce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:24.852870 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7556e27b-cf84-48f1-8c80-b30ba33c94ce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:24.859078 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5e9e3762-3d00-4814-b3e4-6e41f4cf52b0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:24.863982 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5e9e3762-3d00-4814-b3e4-6e41f4cf52b0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e5af107c-5b71-461c-a3e1-deb10425a0f9', 'volume_name': 'volume-e5af107c-5b71-461c-a3e1-deb10425a0f9', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:24.872703 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (85ee33d7-5612-4c4a-8fa9-5360b6c528ec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:24.877528 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (85ee33d7-5612-4c4a-8fa9-5360b6c528ec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:24.884329 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6cbe8e9a-fdfe-4026-becf-31e62b31c843) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:24.886696 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Volume e5af107c-5b71-461c-a3e1-deb10425a0f9: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e5af107c-5b71-461c-a3e1-deb10425a0f9', 'volume_size': 1} Sep 27 10:14:24.887875 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] creating volume 'volume-e5af107c-5b71-461c-a3e1-deb10425a0f9' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:14:24.889326 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:25.086644 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:14:25.087012 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:14:25.091280 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6cbe8e9a-fdfe-4026-becf-31e62b31c843) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e5af107c-5b71-461c-a3e1-deb10425a0f9', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:25.099988 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e3663de2-1f7e-4bd3-ad15-a9ef14de4f32) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:25.122609 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Volume volume-e5af107c-5b71-461c-a3e1-deb10425a0f9 (e5af107c-5b71-461c-a3e1-deb10425a0f9): created successfully Sep 27 10:14:25.130870 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e3663de2-1f7e-4bd3-ad15-a9ef14de4f32) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:25.144652 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Flow 'volume_create_manager' (4d7c0de8-16f6-4e23-bdb9-586bdee4820e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:14:25.169991 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-18444c06-5034-4336-9a92-ceed2bb75689 tempest-AttachVolumeTestJSON-1476802274 None] Created volume successfully. Sep 27 10:14:26.712497 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 req-57880429-1261-4595-8229-40ff88337dc4 tempest-AttachVolumeTestJSON-1476802274 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:14:28.325818 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 req-57880429-1261-4595-8229-40ff88337dc4 tempest-AttachVolumeTestJSON-1476802274 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.613s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:14:28.327472 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 req-57880429-1261-4595-8229-40ff88337dc4 tempest-AttachVolumeTestJSON-1476802274 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e5af107c-5b71-461c-a3e1-deb10425a0f9', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'e5af107c-5b71-461c-a3e1-deb10425a0f9', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:14:28.377392 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 req-57880429-1261-4595-8229-40ff88337dc4 tempest-AttachVolumeTestJSON-1476802274 None] Connection info returned from driver {'name': 'volumes/volume-e5af107c-5b71-461c-a3e1-deb10425a0f9', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'e5af107c-5b71-461c-a3e1-deb10425a0f9', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'ce98d853-45a7-47e2-bd6e-1347bed6311b'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:14:28.477371 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 req-57880429-1261-4595-8229-40ff88337dc4 tempest-AttachVolumeTestJSON-1476802274 None] attachment_update completed successfully. Sep 27 10:14:29.727549 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:14:29.728423 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:29.789675 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:29.869019 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:14:30.901196 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:14:30.930504 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Flow 'volume_create_manager' (9489cbea-e1ea-470b-b7b2-bfb1c71a6331) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:14:30.938778 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7c53ceb3-cbdd-4c73-873b-33d9f48ab27f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:30.962202 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7c53ceb3-cbdd-4c73-873b-33d9f48ab27f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:14:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV357Test-Volume-308494028',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=faa7488c-a863-4d65-b77c-696d3c7aa893,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='35e93ba1c9624e81aeed5f897483b28f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:14:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:14:31Z,use_quota=True,user_id='bdf718f851884452860701c998796495',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:30.968038 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7b4cf1cd-9f45-4667-bc8a-88a4e250c993) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:30.972308 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7b4cf1cd-9f45-4667-bc8a-88a4e250c993) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:30.978259 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23e42719-1116-4075-bcfd-70a8717a98e8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:30.982540 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23e42719-1116-4075-bcfd-70a8717a98e8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'faa7488c-a863-4d65-b77c-696d3c7aa893', 'volume_name': 'volume-faa7488c-a863-4d65-b77c-696d3c7aa893', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:30.988408 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8dcc3980-c9fd-4243-aac3-5b9804c75924) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:30.993138 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8dcc3980-c9fd-4243-aac3-5b9804c75924) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:30.998255 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7cd322ad-a405-4304-bdc2-579cb2105cc4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:30.999495 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Volume faa7488c-a863-4d65-b77c-696d3c7aa893: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-faa7488c-a863-4d65-b77c-696d3c7aa893', 'volume_size': 1} Sep 27 10:14:31.000038 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] creating volume 'volume-faa7488c-a863-4d65-b77c-696d3c7aa893' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:14:31.001469 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:31.090168 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:14:31.092734 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:14:31.097919 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7cd322ad-a405-4304-bdc2-579cb2105cc4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-faa7488c-a863-4d65-b77c-696d3c7aa893', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:31.104886 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1fa7134d-290a-4f0a-86a0-cf8b6826e9b5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:31.118261 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Volume volume-faa7488c-a863-4d65-b77c-696d3c7aa893 (faa7488c-a863-4d65-b77c-696d3c7aa893): created successfully Sep 27 10:14:31.123114 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1fa7134d-290a-4f0a-86a0-cf8b6826e9b5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:31.128566 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Flow 'volume_create_manager' (9489cbea-e1ea-470b-b7b2-bfb1c71a6331) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:14:31.142555 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6cc7cd23-275a-4760-b841-244fd3b5e56e tempest-VolumesTransfersV357Test-1667392102 None] Created volume successfully. Sep 27 10:14:33.059596 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7b0c131e-3768-49f3-b1f8-30facd09ab43 tempest-VolumesTransfersV357Test-1138921126 None] Transfer volume completed successfully. Sep 27 10:14:33.597977 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] Acquiring lock "cinder-faa7488c-a863-4d65-b77c-696d3c7aa893-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:14:33.618601 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] Lock "cinder-faa7488c-a863-4d65-b77c-696d3c7aa893-delete_volume" acquired by "delete_volume" :: waited 0.021s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:14:33.659711 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:33.736937 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:14:33.738599 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] Volume volume-faa7488c-a863-4d65-b77c-696d3c7aa893 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:14:33.750165 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] deleting rbd volume volume-faa7488c-a863-4d65-b77c-696d3c7aa893 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:14:34.094493 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] Created reservations ['75903b5f-010b-4d6b-9a3d-8e928f7b3418', '1879f95d-ca38-454d-8111-e668501cfd6f', 'dd02ce42-6041-430d-8b4c-97a64aa015c8', '54c9d2b3-dcf9-4d11-b059-2eda7c547179'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:14:34.175453 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:34.239842 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:34.321861 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:14:34.332442 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] Deleted volume successfully. Sep 27 10:14:34.360463 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-72d7f60d-a282-45cb-9fbc-5b79e0d07413 tempest-VolumesTransfersV357Test-839459601 None] Lock "cinder-faa7488c-a863-4d65-b77c-696d3c7aa893-delete_volume" released by "delete_volume" :: held 0.741s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:14:35.438323 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:14:35.474766 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Flow 'volume_create_manager' (fd23dadd-bdfe-4e6a-9688-4192bb6ca928) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:14:35.484104 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8850752e-2315-410f-af1b-2a8668a6ef3d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:35.511473 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8850752e-2315-410f-af1b-2a8668a6ef3d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:14:35Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV357Test-Volume-1532632473',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=57b1ce49-46d2-4dea-be79-22acbd5beb7d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='35e93ba1c9624e81aeed5f897483b28f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:14:35Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:14:35Z,use_quota=True,user_id='bdf718f851884452860701c998796495',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:35.523545 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d8999c0a-29dd-4b4e-b1b8-4e5617f05870) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:35.534248 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d8999c0a-29dd-4b4e-b1b8-4e5617f05870) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:35.545791 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (023b27ce-8739-4680-b7f9-e605f550f0d6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:35.561007 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (023b27ce-8739-4680-b7f9-e605f550f0d6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '57b1ce49-46d2-4dea-be79-22acbd5beb7d', 'volume_name': 'volume-57b1ce49-46d2-4dea-be79-22acbd5beb7d', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:35.565817 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (af9407c8-9cc8-466f-8794-10d5d75be269) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:35.570940 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (af9407c8-9cc8-466f-8794-10d5d75be269) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:35.576096 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b30a8502-5853-45ac-9e29-143e098852d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:35.577211 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Volume 57b1ce49-46d2-4dea-be79-22acbd5beb7d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-57b1ce49-46d2-4dea-be79-22acbd5beb7d', 'volume_size': 1} Sep 27 10:14:35.577599 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] creating volume 'volume-57b1ce49-46d2-4dea-be79-22acbd5beb7d' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:14:35.578378 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:35.727777 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:14:35.728234 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:14:35.738802 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b30a8502-5853-45ac-9e29-143e098852d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-57b1ce49-46d2-4dea-be79-22acbd5beb7d', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:35.748505 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9aa8cd54-f1ad-4f5c-839a-3d754c1e0a41) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:35.765452 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Volume volume-57b1ce49-46d2-4dea-be79-22acbd5beb7d (57b1ce49-46d2-4dea-be79-22acbd5beb7d): created successfully Sep 27 10:14:35.771374 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9aa8cd54-f1ad-4f5c-839a-3d754c1e0a41) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:35.779649 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Flow 'volume_create_manager' (fd23dadd-bdfe-4e6a-9688-4192bb6ca928) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:14:35.790289 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5af113f9-7828-4979-9420-4bd4f5247089 tempest-VolumesTransfersV357Test-1667392102 None] Created volume successfully. Sep 27 10:14:36.908527 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] Acquiring lock "cinder-57b1ce49-46d2-4dea-be79-22acbd5beb7d-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:14:36.923989 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] Lock "cinder-57b1ce49-46d2-4dea-be79-22acbd5beb7d-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:14:36.928048 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=51,cinder:INSERT=3,cinder:UPDATE=22,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:36.968097 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:37.039959 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:14:37.040475 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] Volume volume-57b1ce49-46d2-4dea-be79-22acbd5beb7d is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:14:37.060141 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] deleting rbd volume volume-57b1ce49-46d2-4dea-be79-22acbd5beb7d {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:14:37.622008 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] Created reservations ['58df895a-18dd-46ff-8d1e-5854438b60a8', '2b61a4f8-26e9-46eb-813e-b56af54b5bdb', '7bc1814d-112e-470b-9cfa-b457857cecbf', '12c7e21c-cc35-4307-ba26-dd663efb51ae'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:14:37.727969 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:37.781090 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:37.855583 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:14:37.860006 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] Deleted volume successfully. Sep 27 10:14:37.883681 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c6737fe7-a1c8-49c4-99fd-fa8f648ef68c tempest-VolumesTransfersV357Test-839459601 None] Lock "cinder-57b1ce49-46d2-4dea-be79-22acbd5beb7d-delete_volume" released by "delete_volume" :: held 0.957s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:14:46.812861 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:14:46.847485 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Flow 'volume_create_manager' (52b57f7c-151a-4948-9322-9369580feda0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:14:46.858218 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f199bb9f-fe75-4c65-bd5c-a8b0c94ed167) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:46.883586 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f199bb9f-fe75-4c65-bd5c-a8b0c94ed167) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:14:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1089636118',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f4b1d088-fc7d-40b8-9816-61926e74a793,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='27c0d3daa08340d89d288bbf811fece9',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:14:47Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:14:47Z,use_quota=True,user_id='dc369ca971424247b96050e09368a7bb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:46.891534 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f1c3acc8-a462-4ba9-839e-70bbc76226c1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:46.910393 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f1c3acc8-a462-4ba9-839e-70bbc76226c1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:46.919453 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1ca2e7f9-ffbd-4da4-a7fd-22b764db4b2b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:46.924796 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1ca2e7f9-ffbd-4da4-a7fd-22b764db4b2b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f4b1d088-fc7d-40b8-9816-61926e74a793', 'volume_name': 'volume-f4b1d088-fc7d-40b8-9816-61926e74a793', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:46.931467 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fe26df91-b20f-442c-b7a6-24c8df67f6de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:46.936262 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fe26df91-b20f-442c-b7a6-24c8df67f6de) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:46.943993 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0d81e554-34db-4dba-8284-27c41d89fa27) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:46.945577 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Volume f4b1d088-fc7d-40b8-9816-61926e74a793: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f4b1d088-fc7d-40b8-9816-61926e74a793', 'volume_size': 1} Sep 27 10:14:46.946214 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] creating volume 'volume-f4b1d088-fc7d-40b8-9816-61926e74a793' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:14:46.947387 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:14:47.106259 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:14:47.106856 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:14:47.110637 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0d81e554-34db-4dba-8284-27c41d89fa27) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f4b1d088-fc7d-40b8-9816-61926e74a793', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:47.115240 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6421b518-8230-48c1-9d7b-dad64bd67cd7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:14:47.130459 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Volume volume-f4b1d088-fc7d-40b8-9816-61926e74a793 (f4b1d088-fc7d-40b8-9816-61926e74a793): created successfully Sep 27 10:14:47.134242 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6421b518-8230-48c1-9d7b-dad64bd67cd7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:14:47.140078 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Flow 'volume_create_manager' (52b57f7c-151a-4948-9322-9369580feda0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:14:47.164981 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-4ab9a2b3-5d71-4b44-93d7-fbf7d5b972fe tempest-VolumesActionsTest-1551375736 None] Created volume successfully. Sep 27 10:14:56.093408 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=53,cinder:UPDATE=22,cinder:DELETE=4,cinder:INSERT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:06.105581 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=12,cinder:INSERT=2,cinder:UPDATE=14,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:09.966544 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c9b5e4d0-de53-402f-b605-bd4dcaec2fc2 tempest-VolumesActionsTest-1551375736 None] Acquiring lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793" by "attach_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:15:09.991606 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c9b5e4d0-de53-402f-b605-bd4dcaec2fc2 tempest-VolumesActionsTest-1551375736 None] Lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793" acquired by "attach_volume" :: waited 0.024s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:15:10.182767 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c9b5e4d0-de53-402f-b605-bd4dcaec2fc2 tempest-VolumesActionsTest-1551375736 None] Attaching volume f4b1d088-fc7d-40b8-9816-61926e74a793 to instance c74b0db7-5f7b-484b-8d2e-feea845f3264 at mountpoint /dev/sdb on host None. Sep 27 10:15:10.239414 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c9b5e4d0-de53-402f-b605-bd4dcaec2fc2 tempest-VolumesActionsTest-1551375736 None] Attach volume completed successfully. Sep 27 10:15:10.246097 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c9b5e4d0-de53-402f-b605-bd4dcaec2fc2 tempest-VolumesActionsTest-1551375736 None] Lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793" released by "attach_volume" :: held 0.256s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:15:10.497046 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b1fcaf1d-040a-4f50-a267-b87ed38e4686 tempest-VolumesActionsTest-1551375736 None] Acquiring lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793-detach_volume" by "detach_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:15:10.524485 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b1fcaf1d-040a-4f50-a267-b87ed38e4686 tempest-VolumesActionsTest-1551375736 None] Lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793-detach_volume" acquired by "detach_volume" :: waited 0.032s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:15:10.525485 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b1fcaf1d-040a-4f50-a267-b87ed38e4686 tempest-VolumesActionsTest-1551375736 None] Detaching volume f4b1d088-fc7d-40b8-9816-61926e74a793 from instance c74b0db7-5f7b-484b-8d2e-feea845f3264. Sep 27 10:15:10.645127 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b1fcaf1d-040a-4f50-a267-b87ed38e4686 tempest-VolumesActionsTest-1551375736 None] Detach volume completed successfully. Sep 27 10:15:10.697773 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b1fcaf1d-040a-4f50-a267-b87ed38e4686 tempest-VolumesActionsTest-1551375736 None] Lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793-detach_volume" released by "detach_volume" :: held 0.174s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:15:20.597300 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=9,cinder:INSERT=2,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:25.949586 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd req-26277770-589d-49e5-900b-6433b3ba9954 tempest-AttachVolumeTestJSON-1476802274 None] Terminate volume connection completed successfully. Sep 27 10:15:25.950428 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd req-26277770-589d-49e5-900b-6433b3ba9954 tempest-AttachVolumeTestJSON-1476802274 None] Deleting attachment ce98d853-45a7-47e2-bd6e-1347bed6311b. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:15:29.726249 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:15:29.727612 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:15:29.795460 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:15:29.885671 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:15:36.084991 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:36.086696 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=23,cinder:UPDATE=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:40.193953 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-282cbc24-1a50-49ce-a4f2-21c2558ad688 req-09a86674-ac7c-466e-bf05-0fa225807313 tempest-TaggedBootDevicesTest_v242-1873294123 None] Terminate volume connection completed successfully. Sep 27 10:15:40.197080 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-282cbc24-1a50-49ce-a4f2-21c2558ad688 req-09a86674-ac7c-466e-bf05-0fa225807313 tempest-TaggedBootDevicesTest_v242-1873294123 None] Deleting attachment 6866de5d-88ff-4aa7-87c0-95a92e6488bf. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:15:40.409559 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-282cbc24-1a50-49ce-a4f2-21c2558ad688 req-17f0f9d6-4d21-4c8c-8e73-4a625c105df8 tempest-TaggedBootDevicesTest_v242-1873294123 None] Terminate volume connection completed successfully. Sep 27 10:15:40.410121 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-282cbc24-1a50-49ce-a4f2-21c2558ad688 req-17f0f9d6-4d21-4c8c-8e73-4a625c105df8 tempest-TaggedBootDevicesTest_v242-1873294123 None] Deleting attachment 58a4888d-3aa4-46d0-9007-49c5b9a251e8. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:15:40.621776 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-282cbc24-1a50-49ce-a4f2-21c2558ad688 req-0be53c86-c295-4d93-97cc-6ee0204ad3db tempest-TaggedBootDevicesTest_v242-1873294123 None] Terminate volume connection completed successfully. Sep 27 10:15:40.622278 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-282cbc24-1a50-49ce-a4f2-21c2558ad688 req-0be53c86-c295-4d93-97cc-6ee0204ad3db tempest-TaggedBootDevicesTest_v242-1873294123 None] Deleting attachment 957664a1-d47c-4954-a4a1-236b185addb4. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:15:43.090762 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c2966ca1-b964-4f04-b2da-d0a8655301b4 tempest-VolumesActionsTest-1551375736 None] Acquiring lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793" by "attach_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:15:43.101395 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c2966ca1-b964-4f04-b2da-d0a8655301b4 tempest-VolumesActionsTest-1551375736 None] Lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793" acquired by "attach_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:15:43.169963 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c2966ca1-b964-4f04-b2da-d0a8655301b4 tempest-VolumesActionsTest-1551375736 None] Attaching volume f4b1d088-fc7d-40b8-9816-61926e74a793 to instance 168c4335-87f5-4cce-b73b-f525c65f5166 at mountpoint /dev/sdb on host None. Sep 27 10:15:43.207770 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c2966ca1-b964-4f04-b2da-d0a8655301b4 tempest-VolumesActionsTest-1551375736 None] Attach volume completed successfully. Sep 27 10:15:43.214041 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c2966ca1-b964-4f04-b2da-d0a8655301b4 tempest-VolumesActionsTest-1551375736 None] Lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793" released by "attach_volume" :: held 0.113s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:15:43.453026 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-46e97478-c06c-466f-9fa6-4508cf2bcb70 tempest-VolumesActionsTest-1551375736 None] Acquiring lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793-detach_volume" by "detach_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:15:43.464852 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-46e97478-c06c-466f-9fa6-4508cf2bcb70 tempest-VolumesActionsTest-1551375736 None] Lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793-detach_volume" acquired by "detach_volume" :: waited 0.012s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:15:43.465891 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-46e97478-c06c-466f-9fa6-4508cf2bcb70 tempest-VolumesActionsTest-1551375736 None] Detaching volume f4b1d088-fc7d-40b8-9816-61926e74a793 from instance 168c4335-87f5-4cce-b73b-f525c65f5166. Sep 27 10:15:43.521773 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-46e97478-c06c-466f-9fa6-4508cf2bcb70 tempest-VolumesActionsTest-1551375736 None] Detach volume completed successfully. Sep 27 10:15:43.527927 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-46e97478-c06c-466f-9fa6-4508cf2bcb70 tempest-VolumesActionsTest-1551375736 None] Lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793-detach_volume" released by "detach_volume" :: held 0.063s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:15:48.986325 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] Acquiring lock "cinder-1caf2529-26ae-48b5-83d4-b87bb80b3024-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:15:49.020698 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] Lock "cinder-1caf2529-26ae-48b5-83d4-b87bb80b3024-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:15:49.071034 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:15:49.134532 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:15:49.134959 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume volume-1caf2529-26ae-48b5-83d4-b87bb80b3024 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:15:49.156830 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] deleting rbd volume volume-1caf2529-26ae-48b5-83d4-b87bb80b3024 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:15:49.505968 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] Created reservations ['5883bb18-b69e-4afb-a755-1fb3e73ed9e4', '198afc19-54dd-4c1e-8430-b538aeb54186', 'bf6809a3-6f9c-4c3a-a3be-25b77b3102f6', '709dfc51-a0fd-46e2-be25-97be9dc1e130'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:15:49.571789 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:15:49.627002 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:15:49.690720 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:15:49.701384 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] Deleted volume successfully. Sep 27 10:15:49.714973 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-776607fe-a3f7-40cc-b545-e50d9064462a tempest-VolumesActionsTest-1551375736 None] Running cmd (subprocess): rbd export --pool volumes volume-f4b1d088-fc7d-40b8-9816-61926e74a793 /opt/stack/data/cinder/conversion/volume-f4b1d088-fc7d-40b8-9816-61926e74a793-9fac40a8-76d9-43d7-bd15-c994b5a401a6 --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:15:49.746870 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-257cb0ec-5491-4bff-be42-82e17d0098ec tempest-TaggedBootDevicesTest_v242-1873294123 None] Lock "cinder-1caf2529-26ae-48b5-83d4-b87bb80b3024-delete_volume" released by "delete_volume" :: held 0.738s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:15:50.294965 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] Acquiring lock "cinder-1454250c-f356-4053-83eb-5c69c0a805d7-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:15:50.312831 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] Lock "cinder-1454250c-f356-4053-83eb-5c69c0a805d7-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:15:50.413623 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:15:50.497858 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:15:50.498645 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] Volume volume-1454250c-f356-4053-83eb-5c69c0a805d7 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:15:50.525397 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] deleting rbd volume volume-1454250c-f356-4053-83eb-5c69c0a805d7 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:15:50.826713 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-776607fe-a3f7-40cc-b545-e50d9064462a tempest-VolumesActionsTest-1551375736 None] CMD "rbd export --pool volumes volume-f4b1d088-fc7d-40b8-9816-61926e74a793 /opt/stack/data/cinder/conversion/volume-f4b1d088-fc7d-40b8-9816-61926e74a793-9fac40a8-76d9-43d7-bd15-c994b5a401a6 --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.112s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:15:50.836645 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.image_utils [None req-776607fe-a3f7-40cc-b545-e50d9064462a tempest-VolumesActionsTest-1551375736 None] 9fac40a8-76d9-43d7-bd15-c994b5a401a6 was raw, no need to convert to raw {{(pid=138473) upload_volume /opt/stack/cinder/cinder/image/image_utils.py:851}} Sep 27 10:15:51.079438 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] Created reservations ['c86cf7e4-4e84-4b2a-be5c-a2be9c1d7e46', 'f8ba54a1-abd3-43ad-94b7-10b6ef4b37ed', '4f9f6951-8708-491d-a5c5-147b9eace13b', '3f0d440e-7878-4462-bcfe-7f68065508bc'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:15:51.168899 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:15:51.204454 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:15:51.237581 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:15:51.243626 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] Deleted volume successfully. Sep 27 10:15:51.261955 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6d8c6e33-7ce0-467a-b479-bd5f4d0b41ba tempest-TaggedBootDevicesTest_v242-1873294123 None] Lock "cinder-1454250c-f356-4053-83eb-5c69c0a805d7-delete_volume" released by "delete_volume" :: held 0.949s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:15:51.588752 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] Acquiring lock "cinder-52fbc488-d91e-4ac4-b7b6-28284e31df30-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:15:51.629554 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] Lock "cinder-52fbc488-d91e-4ac4-b7b6-28284e31df30-delete_volume" acquired by "delete_volume" :: waited 0.041s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:15:51.760061 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:15:51.890948 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:15:51.905641 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] deleting rbd volume volume-52fbc488-d91e-4ac4-b7b6-28284e31df30 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:15:52.628758 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:15:52.682300 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] deleting parent snapshot volume-52fbc488-d91e-4ac4-b7b6-28284e31df30.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:15:53.570366 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] Created reservations ['6c5439d0-f533-4f00-a99d-d1b8d87fab3d', '5f1c9832-16cd-4add-8b35-e8130246aebf', '2375d0c9-3970-4c9e-ace2-459451f72142', 'e434120c-134d-451f-9aa1-1c4323814aed'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:15:53.622922 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=65,cinder:UPDATE=29,cinder:INSERT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:53.642932 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:15:53.686975 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:15:53.716777 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:15:53.721920 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] Deleted volume successfully. Sep 27 10:15:53.741231 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-36768275-10cd-4983-ad6d-c3783a8d3fbb tempest-TaggedBootDevicesTest_v242-1873294123 None] Lock "cinder-52fbc488-d91e-4ac4-b7b6-28284e31df30-delete_volume" released by "delete_volume" :: held 2.112s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:16:03.729395 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=50,cinder:INSERT=8,cinder:UPDATE=27,cinder:DELETE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:16.095330 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:26.085448 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:29.724314 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:16:29.725294 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:29.833484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:29.939324 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:16:31.839611 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-776607fe-a3f7-40cc-b545-e50d9064462a tempest-VolumesActionsTest-1551375736 None] Uploaded volume to glance image-id: 9fac40a8-76d9-43d7-bd15-c994b5a401a6. {{(pid=138473) copy_volume_to_image /opt/stack/cinder/cinder/volume/manager.py:1791}} Sep 27 10:16:31.878526 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-776607fe-a3f7-40cc-b545-e50d9064462a tempest-VolumesActionsTest-1551375736 None] Copy volume to image completed successfully. Sep 27 10:16:34.438518 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] Acquiring lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:16:34.460343 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] Lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793-delete_volume" acquired by "delete_volume" :: waited 0.025s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:16:34.566473 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:34.688342 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:16:34.688874 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] Volume volume-f4b1d088-fc7d-40b8-9816-61926e74a793 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:16:34.698334 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] deleting rbd volume volume-f4b1d088-fc7d-40b8-9816-61926e74a793 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:16:35.591462 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] Created reservations ['4d1c91a9-ced9-44a1-9bd6-dd716addd2ef', 'ce6e85cb-60a3-4ae2-b540-dcd959f1bf2f', '1399218f-77d4-4896-9bf2-9561133da06b', '048932ac-1245-4976-9e48-208116ab81ba'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:16:35.781290 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:35.875355 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:35.966321 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:16:35.973974 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] Deleted volume successfully. Sep 27 10:16:35.995689 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5dfa1f0f-2133-492e-a223-359812d91418 tempest-VolumesActionsTest-1551375736 None] Lock "cinder-f4b1d088-fc7d-40b8-9816-61926e74a793-delete_volume" released by "delete_volume" :: held 1.536s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:16:46.085448 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=16,cinder:UPDATE=8,cinder:INSERT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:46.462240 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=18,cinder:UPDATE=13,cinder:INSERT=2,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:46.474374 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:16:46.509089 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Flow 'volume_create_manager' (25325781-38ce-47c3-a746-85efb172a5de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:16:46.540161 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c7f16fe0-a94a-40c5-a7e3-495c3fd5da94) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:46.572794 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c7f16fe0-a94a-40c5-a7e3-495c3fd5da94) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:16:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesExtendTest-Volume-10551736',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=04117117-a2f7-4a62-81ff-f860056b0d87,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='466f9ae4f69f4ba6a9484b5a93bb6284',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:16:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:16:46Z,use_quota=True,user_id='5a34669b510b41eea8b10ba3c9f650ef',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:46.579066 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f704c1a8-d6fc-489d-8f3f-f19c47c438e3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:46.583028 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f704c1a8-d6fc-489d-8f3f-f19c47c438e3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:46.589459 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c5830ec-b75d-4fbc-99cb-b53e11ab9f7e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:46.720383 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c5830ec-b75d-4fbc-99cb-b53e11ab9f7e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '04117117-a2f7-4a62-81ff-f860056b0d87', 'volume_name': 'volume-04117117-a2f7-4a62-81ff-f860056b0d87', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:46.734036 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (21b93b8b-4eab-4a1b-b40c-9fd43b347572) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:46.744004 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (21b93b8b-4eab-4a1b-b40c-9fd43b347572) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:46.755714 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c88b0b80-34ee-452f-879d-7bbda2847977) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:46.757200 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Volume 04117117-a2f7-4a62-81ff-f860056b0d87: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-04117117-a2f7-4a62-81ff-f860056b0d87', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:16:46.757784 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Cloning 04117117-a2f7-4a62-81ff-f860056b0d87 from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:16:46.760558 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:16:46.777568 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.018s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:16:46.814828 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 16, 46, 787774)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:16:46.815851 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:16:46.816457 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:16:46.823498 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.046s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:16:46.824865 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:16:46.853428 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 16, 46, 842425)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:16:46.854227 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:16:46.855026 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:16:46.875127 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:46.987112 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] creating snapshot='volume-04117117-a2f7-4a62-81ff-f860056b0d87.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:16:47.698254 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-04117117-a2f7-4a62-81ff-f860056b0d87.clone_snap' to 'volume-04117117-a2f7-4a62-81ff-f860056b0d87' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:16:47.948348 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:16:47.951435 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:16:47.952143 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:16:47.992785 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:16:47.994656 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Marking volume 04117117-a2f7-4a62-81ff-f860056b0d87 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:16:48.012504 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 04117117-a2f7-4a62-81ff-f860056b0d87. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:16:48.015160 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Creating volume glance metadata for volume 04117117-a2f7-4a62-81ff-f860056b0d87 backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:16:48.156904 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c88b0b80-34ee-452f-879d-7bbda2847977) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-04117117-a2f7-4a62-81ff-f860056b0d87', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:48.188259 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (580deb69-9eeb-4aa7-899e-95b1a46ee8f5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:48.205663 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Volume volume-04117117-a2f7-4a62-81ff-f860056b0d87 (04117117-a2f7-4a62-81ff-f860056b0d87): created successfully Sep 27 10:16:48.228672 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (580deb69-9eeb-4aa7-899e-95b1a46ee8f5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:48.240593 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Flow 'volume_create_manager' (25325781-38ce-47c3-a746-85efb172a5de) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:16:48.263920 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b7c2fc8f-61be-4da0-9bf3-69f9ad56bbf6 tempest-VolumesExtendTest-535108842 None] Created volume successfully. Sep 27 10:16:48.933982 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-63e7e95c-61b9-48ae-8480-6f1c8bcd2b8f tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:49.164274 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-63e7e95c-61b9-48ae-8480-6f1c8bcd2b8f tempest-VolumesExtendTest-535108842 None] Extend volume from 1 GB to 2 GB. {{(pid=138473) extend_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:2012}} Sep 27 10:16:49.223896 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-63e7e95c-61b9-48ae-8480-6f1c8bcd2b8f tempest-VolumesExtendTest-535108842 None] Extend volume completed successfully. Sep 27 10:16:50.767847 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:16:50.810173 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Flow 'volume_create_manager' (68a6e35f-457f-4a58-9a1d-58fb94755a70) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:16:50.837393 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (277b1bca-7b57-4470-b0e3-8c28dd0ec451) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:50.864624 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (277b1bca-7b57-4470-b0e3-8c28dd0ec451) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:16:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesExtendTest-Volume-100167805',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=76faa8b5-dc12-4078-9483-01dcc5e41d20,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='466f9ae4f69f4ba6a9484b5a93bb6284',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:16:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:16:51Z,use_quota=True,user_id='5a34669b510b41eea8b10ba3c9f650ef',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:50.871317 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f55a2088-237f-4ed6-b470-84e1f4cab810) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:50.876137 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f55a2088-237f-4ed6-b470-84e1f4cab810) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:50.882186 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c4543d0f-898e-4215-8ce5-62aea5136f47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:50.886852 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c4543d0f-898e-4215-8ce5-62aea5136f47) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '76faa8b5-dc12-4078-9483-01dcc5e41d20', 'volume_name': 'volume-76faa8b5-dc12-4078-9483-01dcc5e41d20', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:50.895800 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0c198a86-8786-4736-b8b4-76fd7ed92e05) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:50.900529 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0c198a86-8786-4736-b8b4-76fd7ed92e05) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:50.908415 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6ec80b60-d8ca-457c-8459-e4ced83df416) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:50.909870 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Volume 76faa8b5-dc12-4078-9483-01dcc5e41d20: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-76faa8b5-dc12-4078-9483-01dcc5e41d20', 'volume_size': 1} Sep 27 10:16:50.910397 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] creating volume 'volume-76faa8b5-dc12-4078-9483-01dcc5e41d20' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:16:50.911351 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:51.030422 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:16:51.031305 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:16:51.035921 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6ec80b60-d8ca-457c-8459-e4ced83df416) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-76faa8b5-dc12-4078-9483-01dcc5e41d20', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:51.043638 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fcfc2ba3-84e0-4576-91ed-90fee26fc626) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:16:51.069322 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Volume volume-76faa8b5-dc12-4078-9483-01dcc5e41d20 (76faa8b5-dc12-4078-9483-01dcc5e41d20): created successfully Sep 27 10:16:51.074530 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fcfc2ba3-84e0-4576-91ed-90fee26fc626) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:16:51.106060 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Flow 'volume_create_manager' (68a6e35f-457f-4a58-9a1d-58fb94755a70) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:16:51.150508 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c0d7c414-7e01-44b5-837a-2cf03d9a4bc6 tempest-VolumesExtendTest-535108842 None] Created volume successfully. Sep 27 10:16:52.222513 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-168b0747-b5f4-4ef8-9d41-e08c44e58f68 tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:52.665428 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-168b0747-b5f4-4ef8-9d41-e08c44e58f68 tempest-VolumesExtendTest-535108842 None] Create snapshot completed successfully Sep 27 10:16:53.549865 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4e00e91a-ee15-4c71-b67c-258b3df9a5c3 tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:53.755718 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4e00e91a-ee15-4c71-b67c-258b3df9a5c3 tempest-VolumesExtendTest-535108842 None] Extend volume from 1 GB to 2 GB. {{(pid=138473) extend_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:2012}} Sep 27 10:16:53.824933 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-4e00e91a-ee15-4c71-b67c-258b3df9a5c3 tempest-VolumesExtendTest-535108842 None] Extend volume completed successfully. Sep 27 10:16:54.926723 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c589d146-a126-4f56-9959-10f738e71c2e tempest-VolumesExtendTest-535108842 None] Acquiring lock "cinder-3ee0b5bb-1256-408e-98eb-37631dcde587-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:16:54.941917 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c589d146-a126-4f56-9959-10f738e71c2e tempest-VolumesExtendTest-535108842 None] Lock "cinder-3ee0b5bb-1256-408e-98eb-37631dcde587-delete_snapshot" acquired by "delete_snapshot" :: waited 0.016s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:16:54.943652 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c589d146-a126-4f56-9959-10f738e71c2e tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:55.714524 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-c589d146-a126-4f56-9959-10f738e71c2e tempest-VolumesExtendTest-535108842 None] Deprecated: Default quota for resource: snapshots_ceph is set by the default quota flag: quota_snapshots_ceph, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:16:55.735648 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-c589d146-a126-4f56-9959-10f738e71c2e tempest-VolumesExtendTest-535108842 None] Created reservations ['bcd231e7-2c1e-44bd-ab87-4e7c62316d03', '40f74322-5522-4411-b204-b77ce604b258', '1b2208b2-5b9a-4a91-8971-4d93b11da10e', '758b5f8a-447e-4d58-b301-428e6696db29'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:16:55.788538 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c589d146-a126-4f56-9959-10f738e71c2e tempest-VolumesExtendTest-535108842 None] Delete snapshot completed successfully. Sep 27 10:16:55.797142 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c589d146-a126-4f56-9959-10f738e71c2e tempest-VolumesExtendTest-535108842 None] Lock "cinder-3ee0b5bb-1256-408e-98eb-37631dcde587-delete_snapshot" released by "delete_snapshot" :: held 0.855s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:16:56.145588 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] Acquiring lock "cinder-76faa8b5-dc12-4078-9483-01dcc5e41d20-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:16:56.158519 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] Lock "cinder-76faa8b5-dc12-4078-9483-01dcc5e41d20-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:16:56.200780 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:56.256062 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:16:56.256804 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] Volume volume-76faa8b5-dc12-4078-9483-01dcc5e41d20 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:16:56.262072 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] deleting rbd volume volume-76faa8b5-dc12-4078-9483-01dcc5e41d20 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:16:56.627460 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] Created reservations ['fa95cdb2-7e67-4f5f-aea3-c417cfe27030', '120588f1-a6aa-4d94-b51a-0269586da499', 'cba58778-e786-446e-a914-35240088aab6', 'c8b11ef6-2ce7-4468-bbcb-8e2f09170c05'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:16:56.685696 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:56.780812 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:56.836438 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:16:56.840866 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] Deleted volume successfully. Sep 27 10:16:56.864144 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8f168389-4d15-4a49-a7df-44a601c84d60 tempest-VolumesExtendTest-535108842 None] Lock "cinder-76faa8b5-dc12-4078-9483-01dcc5e41d20-delete_volume" released by "delete_volume" :: held 0.706s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:16:57.418889 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] Acquiring lock "cinder-04117117-a2f7-4a62-81ff-f860056b0d87-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:16:57.462290 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] Lock "cinder-04117117-a2f7-4a62-81ff-f860056b0d87-delete_volume" acquired by "delete_volume" :: waited 0.044s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:16:57.563821 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:57.710447 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:16:57.719413 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] deleting rbd volume volume-04117117-a2f7-4a62-81ff-f860056b0d87 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:16:58.035999 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:16:58.077156 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] deleting parent snapshot volume-04117117-a2f7-4a62-81ff-f860056b0d87.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:16:58.778322 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=62,cinder:UPDATE=28,cinder:INSERT=9,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:58.814722 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] Created reservations ['a318d6d7-1395-4e7b-8508-2776bb53623d', '3b639787-9344-4111-a47f-ac862bf951e8', 'e47a2d74-d5ef-4b98-8501-13245d8ad9c0', 'aa754504-22d0-4427-82b2-dcfd907b9249'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:16:58.822748 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=59,cinder:INSERT=15,cinder:UPDATE=23,cinder:DELETE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:58.878063 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:58.977860 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:16:59.076674 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:16:59.088735 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] Deleted volume successfully. Sep 27 10:16:59.106794 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-29d647e5-cfa0-4ece-95e2-d95a6d340f18 tempest-VolumesExtendTest-535108842 None] Lock "cinder-04117117-a2f7-4a62-81ff-f860056b0d87-delete_volume" released by "delete_volume" :: held 1.644s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:17:09.096603 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:16.107029 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=14,cinder:SELECT=7 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:23.789212 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-01e54783-f044-4ff1-a3ec-bcddb14b13d8 tempest-GroupsTest-900404156 None] Group tempest-Group1-170718194: creating Sep 27 10:17:23.848913 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-01e54783-f044-4ff1-a3ec-bcddb14b13d8 tempest-GroupsTest-900404156 None] Group tempest-Group1-170718194: created successfully Sep 27 10:17:23.863523 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-01e54783-f044-4ff1-a3ec-bcddb14b13d8 tempest-GroupsTest-900404156 None] Create group completed successfully. Sep 27 10:17:25.061056 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-879013b8-e0a3-41ab-b867-29d68355a489 tempest-GroupsTest-900404156 None] Group tempest-Group2-1597046115: creating Sep 27 10:17:25.096050 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-879013b8-e0a3-41ab-b867-29d68355a489 tempest-GroupsTest-900404156 None] Group tempest-Group2-1597046115: created successfully Sep 27 10:17:25.108478 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-879013b8-e0a3-41ab-b867-29d68355a489 tempest-GroupsTest-900404156 None] Create group completed successfully. Sep 27 10:17:26.953723 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:17:26.998932 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Flow 'volume_create_manager' (237ad33a-98cb-46b2-912b-bca2cde02d5b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:27.011515 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (27c808d7-90a2-47b8-9498-46dc00265dac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:27.043196 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (27c808d7-90a2-47b8-9498-46dc00265dac) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:17:27Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-volume-565107973',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=5bc8586c-6e2a-4267-82a5-20bc8fc958c8,host='ncdv-master-1@ceph#ceph',id=59846f8c-c563-4999-a3b5-80ace6e8fada,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='64dd9a15966148b88e42e312d29f0e55',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:17:27Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:17:27Z,use_quota=True,user_id='eea2a6c4a4f049ebba996287df6ef7a5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(d3e10627-9fad-48b2-bca5-8770954c4ce3),volume_type_id=d3e10627-9fad-48b2-bca5-8770954c4ce3)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:27.054411 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2822164c-b297-44bc-b9cb-1b6c763bcbe4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:27.065376 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2822164c-b297-44bc-b9cb-1b6c763bcbe4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:27.072611 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b6c8d1b1-fbe5-49fa-ba96-1cf3a6ef2243) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:27.077375 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b6c8d1b1-fbe5-49fa-ba96-1cf3a6ef2243) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '59846f8c-c563-4999-a3b5-80ace6e8fada', 'volume_name': 'volume-59846f8c-c563-4999-a3b5-80ace6e8fada', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:27.083128 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7c2bd003-32dc-413b-b6c7-5f8363ff0634) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:27.089463 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7c2bd003-32dc-413b-b6c7-5f8363ff0634) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:27.094054 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (05dcd8ee-2657-48e4-bd1a-4180c93e5823) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:27.095376 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Volume 59846f8c-c563-4999-a3b5-80ace6e8fada: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-59846f8c-c563-4999-a3b5-80ace6e8fada', 'volume_size': 1} Sep 27 10:17:27.095780 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] creating volume 'volume-59846f8c-c563-4999-a3b5-80ace6e8fada' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:17:27.096582 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:27.246836 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:17:27.247213 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:17:27.251036 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (05dcd8ee-2657-48e4-bd1a-4180c93e5823) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-59846f8c-c563-4999-a3b5-80ace6e8fada', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:27.255382 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8fd99220-6862-403f-b27d-b769ed1c87b8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:27.289030 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Volume volume-59846f8c-c563-4999-a3b5-80ace6e8fada (59846f8c-c563-4999-a3b5-80ace6e8fada): created successfully Sep 27 10:17:27.295265 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8fd99220-6862-403f-b27d-b769ed1c87b8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:27.305549 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Flow 'volume_create_manager' (237ad33a-98cb-46b2-912b-bca2cde02d5b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:27.319014 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0d3bd9ce-9954-4fd3-8684-99efd820bb62 tempest-GroupsTest-900404156 None] Created volume successfully. Sep 27 10:17:28.959350 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:29.035327 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:17:29.036176 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] Volume volume-59846f8c-c563-4999-a3b5-80ace6e8fada is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:17:29.043443 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] deleting rbd volume volume-59846f8c-c563-4999-a3b5-80ace6e8fada {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:17:29.502277 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] Created reservations ['049d084d-45eb-4744-a941-8c5d81d17d0c'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:17:29.535472 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-744621196 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-744621196, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:17:29.537528 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-744621196 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-744621196, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:17:29.560448 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] Created reservations ['dff2d65e-47ed-4fcc-bda3-b71434652881', '906bd386-bad5-412d-9789-a5c62c65ad6b', 'c5579f4a-19e8-4863-baf5-14aa4f3a2f15', 'c07bcb59-0fed-4d69-b982-eeac9ffc509c'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:17:29.696330 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:29.764974 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:17:29.767683 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:29.838365 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:29.909561 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:17:29.918284 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1b00942e-3f8a-4cfb-b0b9-4af5694b8ca9 tempest-GroupsTest-900404156 None] Delete group completed successfully. Sep 27 10:17:29.919709 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:29.987806 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:17:30.348706 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-df0ff943-99e5-43f8-a72d-955fd7a87522 tempest-GroupsTest-900404156 None] Created reservations ['417eb30b-788b-416b-b1be-94bb8e4a49cd'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:17:30.413381 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-df0ff943-99e5-43f8-a72d-955fd7a87522 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:30.501370 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-df0ff943-99e5-43f8-a72d-955fd7a87522 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:30.583833 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-df0ff943-99e5-43f8-a72d-955fd7a87522 tempest-GroupsTest-900404156 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:17:30.594357 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-df0ff943-99e5-43f8-a72d-955fd7a87522 tempest-GroupsTest-900404156 None] Delete group completed successfully. Sep 27 10:17:32.093745 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-fb2c2905-fdc5-4e3f-b63d-7d619fa1f8c1 tempest-GroupsTest-900404156 None] Group tempest-GroupsTest-Group-285542535: creating Sep 27 10:17:32.125227 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-fb2c2905-fdc5-4e3f-b63d-7d619fa1f8c1 tempest-GroupsTest-900404156 None] Group tempest-GroupsTest-Group-285542535: created successfully Sep 27 10:17:32.137753 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-fb2c2905-fdc5-4e3f-b63d-7d619fa1f8c1 tempest-GroupsTest-900404156 None] Create group completed successfully. Sep 27 10:17:33.779211 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:17:33.847550 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Flow 'volume_create_manager' (f73b8fb5-7207-4719-b5f6-f5e10f74c684) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:33.855692 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (454a1b96-3089-459e-a9bf-d72b46da1224) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:33.889356 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (454a1b96-3089-459e-a9bf-d72b46da1224) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:17:33Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsTest-Volume-1793045726',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=f36ad1ba-f157-4e81-8736-b31651cd6a06,host='ncdv-master-1@ceph#ceph',id=857f8955-127a-4e3b-9d65-88d4e49cb1f6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='64dd9a15966148b88e42e312d29f0e55',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:17:34Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:17:34Z,use_quota=True,user_id='eea2a6c4a4f049ebba996287df6ef7a5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(67d73ebd-d209-40d2-b6b4-67c279d6e120),volume_type_id=67d73ebd-d209-40d2-b6b4-67c279d6e120)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:33.912302 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3bf16ad3-0109-4e43-9bb5-21a9d125a922) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:33.918137 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3bf16ad3-0109-4e43-9bb5-21a9d125a922) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:33.926617 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bb2bd2d1-04aa-4e58-95c0-8f9341e11615) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:33.931600 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bb2bd2d1-04aa-4e58-95c0-8f9341e11615) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '857f8955-127a-4e3b-9d65-88d4e49cb1f6', 'volume_name': 'volume-857f8955-127a-4e3b-9d65-88d4e49cb1f6', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:33.945738 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2fbe6fc2-f27c-475c-a940-7714a010dd31) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:33.949605 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2fbe6fc2-f27c-475c-a940-7714a010dd31) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:33.965673 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a5ef16ef-0467-4df6-ad4c-def466d4bdfa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:33.971250 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Volume 857f8955-127a-4e3b-9d65-88d4e49cb1f6: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-857f8955-127a-4e3b-9d65-88d4e49cb1f6', 'volume_size': 1} Sep 27 10:17:33.972127 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] creating volume 'volume-857f8955-127a-4e3b-9d65-88d4e49cb1f6' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:17:33.975520 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:34.137261 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:17:34.138361 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:17:34.142245 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a5ef16ef-0467-4df6-ad4c-def466d4bdfa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-857f8955-127a-4e3b-9d65-88d4e49cb1f6', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:34.151392 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (94632fc7-e938-4a69-a70f-b4dd059b9da4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:34.177834 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Volume volume-857f8955-127a-4e3b-9d65-88d4e49cb1f6 (857f8955-127a-4e3b-9d65-88d4e49cb1f6): created successfully Sep 27 10:17:34.201391 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (94632fc7-e938-4a69-a70f-b4dd059b9da4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:34.210437 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Flow 'volume_create_manager' (f73b8fb5-7207-4719-b5f6-f5e10f74c684) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:34.245351 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5c9ca7b0-142f-4dc7-a112-2fde91e2e0d5 tempest-GroupsTest-900404156 None] Created volume successfully. Sep 27 10:17:35.369439 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:17:35.395138 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Flow 'volume_create_manager' (5507f112-7ffc-4799-855c-ec594cd4ca47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:35.403688 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f197a419-a21b-40e3-be0a-265bf533f5c1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:35.430230 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f197a419-a21b-40e3-be0a-265bf533f5c1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:17:35Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsTest-Volume-1529102667',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=f36ad1ba-f157-4e81-8736-b31651cd6a06,host='ncdv-master-1@ceph#ceph',id=0afc3992-86d8-4696-85a9-20758ddb6313,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='64dd9a15966148b88e42e312d29f0e55',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:17:35Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:17:35Z,use_quota=True,user_id='eea2a6c4a4f049ebba996287df6ef7a5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(67d73ebd-d209-40d2-b6b4-67c279d6e120),volume_type_id=67d73ebd-d209-40d2-b6b4-67c279d6e120)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:35.437992 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c6f4bf73-8680-4b2b-9eb4-bcbb7e05b942) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:35.446780 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c6f4bf73-8680-4b2b-9eb4-bcbb7e05b942) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:35.454333 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b8b7c7fb-323b-48ba-8394-5e208cbe7e67) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:35.459615 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b8b7c7fb-323b-48ba-8394-5e208cbe7e67) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '0afc3992-86d8-4696-85a9-20758ddb6313', 'volume_name': 'volume-0afc3992-86d8-4696-85a9-20758ddb6313', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:35.464957 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1077c92e-2244-43bd-98ac-e5ec0488e955) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:35.468873 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1077c92e-2244-43bd-98ac-e5ec0488e955) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:35.473495 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (57f0e5ad-623f-4118-8346-761ee2704a14) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:35.474832 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Volume 0afc3992-86d8-4696-85a9-20758ddb6313: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-0afc3992-86d8-4696-85a9-20758ddb6313', 'volume_size': 1} Sep 27 10:17:35.475626 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] creating volume 'volume-0afc3992-86d8-4696-85a9-20758ddb6313' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:17:35.476802 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:35.629071 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:17:35.629435 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:17:35.634363 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (57f0e5ad-623f-4118-8346-761ee2704a14) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-0afc3992-86d8-4696-85a9-20758ddb6313', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:35.641181 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6a9b5ca4-adcb-40ba-9446-9579377ec9c1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:35.677902 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Volume volume-0afc3992-86d8-4696-85a9-20758ddb6313 (0afc3992-86d8-4696-85a9-20758ddb6313): created successfully Sep 27 10:17:35.701647 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6a9b5ca4-adcb-40ba-9446-9579377ec9c1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:35.707822 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Flow 'volume_create_manager' (5507f112-7ffc-4799-855c-ec594cd4ca47) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:35.728646 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-bf079137-9a4a-4ed7-96ef-72c383e5bfe1 tempest-GroupsTest-900404156 None] Created volume successfully. Sep 27 10:17:36.784021 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-812b1427-89aa-4549-ae6f-c34fa38d6889 tempest-GroupsTest-900404156 None] Update group completed successfully. Sep 27 10:17:38.344048 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0b17ab17-b905-4a28-9f55-0b31d3717716 tempest-GroupsTest-900404156 None] Update group completed successfully. Sep 27 10:17:39.439533 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=72,cinder:UPDATE=23,cinder:INSERT=1,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:39.469848 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-314b3619-f581-44a7-b147-b4f6a33bdaea req-77dd1d84-aa12-4a85-a7a0-918106c28a69 tempest-TaggedBootDevicesTest-1222598264 None] Terminate volume connection completed successfully. Sep 27 10:17:39.470582 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-314b3619-f581-44a7-b147-b4f6a33bdaea req-77dd1d84-aa12-4a85-a7a0-918106c28a69 tempest-TaggedBootDevicesTest-1222598264 None] Deleting attachment d2678885-9f57-4e5f-b61f-1b2df0c41832. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:17:39.731988 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-314b3619-f581-44a7-b147-b4f6a33bdaea req-6707f070-b5c0-4065-8419-1ec781646b71 tempest-TaggedBootDevicesTest-1222598264 None] Terminate volume connection completed successfully. Sep 27 10:17:39.741036 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-314b3619-f581-44a7-b147-b4f6a33bdaea req-6707f070-b5c0-4065-8419-1ec781646b71 tempest-TaggedBootDevicesTest-1222598264 None] Deleting attachment cf48e7d0-ac5b-44ad-ab11-08b4099b156c. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:17:40.021859 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=68,cinder:UPDATE=25,cinder:DELETE=2,cinder:INSERT=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:40.052473 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-314b3619-f581-44a7-b147-b4f6a33bdaea req-304b42cf-04f8-43bb-a205-abed67cab80d tempest-TaggedBootDevicesTest-1222598264 None] Terminate volume connection completed successfully. Sep 27 10:17:40.054187 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-314b3619-f581-44a7-b147-b4f6a33bdaea req-304b42cf-04f8-43bb-a205-abed67cab80d tempest-TaggedBootDevicesTest-1222598264 None] Deleting attachment b96037de-e63f-49eb-92f2-463bd277e3e6. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:17:40.271834 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:40.371274 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:17:40.371743 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] Volume volume-0afc3992-86d8-4696-85a9-20758ddb6313 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:17:40.382211 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] deleting rbd volume volume-0afc3992-86d8-4696-85a9-20758ddb6313 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:17:40.688518 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:40.766867 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:17:40.767507 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] Volume volume-857f8955-127a-4e3b-9d65-88d4e49cb1f6 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:17:40.793939 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] deleting rbd volume volume-857f8955-127a-4e3b-9d65-88d4e49cb1f6 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:17:40.900431 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:17:40.924376 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Flow 'volume_create_manager' (968d122c-34e7-457b-9166-794e3e0637f8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:40.948948 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d0c280bb-d3d6-4a63-9e8b-108bf3ca3877) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:40.976130 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d0c280bb-d3d6-4a63-9e8b-108bf3ca3877) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:17:41Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-30271830',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3bc57786-1606-4e4f-81f1-868696186ba0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e2e00c90216b4d1b80ac7892ee23550f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:17:41Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:17:41Z,use_quota=True,user_id='279d3662f60c4a979c22ee3e8fa92865',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:40.984783 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1084d1f8-6822-4bfa-b4fc-708e78102816) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:40.989757 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1084d1f8-6822-4bfa-b4fc-708e78102816) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:40.996626 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (daac328a-1ec6-46dd-8ed9-652d2ffcab98) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:41.015274 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (daac328a-1ec6-46dd-8ed9-652d2ffcab98) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '3bc57786-1606-4e4f-81f1-868696186ba0', 'volume_name': 'volume-3bc57786-1606-4e4f-81f1-868696186ba0', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:41.026275 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7890e0ab-5504-45ae-8774-db18ae7a5dc5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:41.032218 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7890e0ab-5504-45ae-8774-db18ae7a5dc5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:41.039577 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (50701bb2-ffc4-4b50-aa8f-c258ad9e607d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:41.040910 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Volume 3bc57786-1606-4e4f-81f1-868696186ba0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-3bc57786-1606-4e4f-81f1-868696186ba0', 'volume_size': 1} Sep 27 10:17:41.041499 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] creating volume 'volume-3bc57786-1606-4e4f-81f1-868696186ba0' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:17:41.042787 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:41.134898 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:17:41.137097 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:17:41.143785 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (50701bb2-ffc4-4b50-aa8f-c258ad9e607d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3bc57786-1606-4e4f-81f1-868696186ba0', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:41.152594 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (96935b7b-72fe-4cc1-8481-cdac7b355e9a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:41.177742 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Volume volume-3bc57786-1606-4e4f-81f1-868696186ba0 (3bc57786-1606-4e4f-81f1-868696186ba0): created successfully Sep 27 10:17:41.182375 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (96935b7b-72fe-4cc1-8481-cdac7b355e9a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:41.189675 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Flow 'volume_create_manager' (968d122c-34e7-457b-9166-794e3e0637f8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:41.223097 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b4a37b60-fc1c-4a74-a57c-eb7e834e7f04 tempest-AttachVolumeTestJSON-1476802274 None] Created volume successfully. Sep 27 10:17:41.302162 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] Created reservations ['634fdb71-2cb7-49b3-9c75-fca24bf7e286'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:17:41.341660 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-311321702 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-311321702, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:17:41.343002 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-311321702 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-311321702, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:17:41.369249 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] Created reservations ['d124d2f1-54de-4dba-8aeb-681902fe178d', 'e026097c-b382-4be3-85f2-414e46a300b4', 'f0f25185-345a-4aa1-a924-ee86f6a9d2d7', '2a21d2a3-5c30-4fdf-97e5-2cd19323867b'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:17:41.529087 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] Created reservations ['711238d4-a47d-4c0f-a685-c5b6bd094e8e', '6b9d81d8-c7eb-4e4d-a3b0-39a6375d3b40', 'e9e39172-8067-46e2-91e2-f6c3c2d1c149', 'fd524a08-1563-446c-9523-1370db9461b4'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:17:41.692050 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:41.752534 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:41.799182 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:17:41.807787 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-777c3795-fa14-44f0-ba2e-a98947d49f24 tempest-GroupsTest-900404156 None] Delete group completed successfully. Sep 27 10:17:43.321288 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-105a79ae-22ae-4029-b776-0e5331134b29 req-090b64d5-c7c3-459f-a088-3a6b61295d2d tempest-AttachVolumeTestJSON-1476802274 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:17:45.220691 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-105a79ae-22ae-4029-b776-0e5331134b29 req-090b64d5-c7c3-459f-a088-3a6b61295d2d tempest-AttachVolumeTestJSON-1476802274 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.899s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:17:45.222546 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-105a79ae-22ae-4029-b776-0e5331134b29 req-090b64d5-c7c3-459f-a088-3a6b61295d2d tempest-AttachVolumeTestJSON-1476802274 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-3bc57786-1606-4e4f-81f1-868696186ba0', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '3bc57786-1606-4e4f-81f1-868696186ba0', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:17:45.281432 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-105a79ae-22ae-4029-b776-0e5331134b29 req-090b64d5-c7c3-459f-a088-3a6b61295d2d tempest-AttachVolumeTestJSON-1476802274 None] Connection info returned from driver {'name': 'volumes/volume-3bc57786-1606-4e4f-81f1-868696186ba0', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '3bc57786-1606-4e4f-81f1-868696186ba0', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '3c1af1a2-c21d-4da4-9546-999d9bce70fc'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:17:45.424640 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-105a79ae-22ae-4029-b776-0e5331134b29 req-090b64d5-c7c3-459f-a088-3a6b61295d2d tempest-AttachVolumeTestJSON-1476802274 None] attachment_update completed successfully. Sep 27 10:17:48.860554 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:17:48.925627 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Flow 'volume_create_manager' (0b3a3339-9e6c-4a19-bbf6-bf0a0eeddd09) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:48.956230 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0766429d-5ac5-4026-a37a-6cc33b207e1c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:48.994718 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0766429d-5ac5-4026-a37a-6cc33b207e1c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:17:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-2114782136',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d8ac5cbc-2d8c-4e85-a955-ecde664d63a4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e2e00c90216b4d1b80ac7892ee23550f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:17:49Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:17:49Z,use_quota=True,user_id='279d3662f60c4a979c22ee3e8fa92865',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:49.007719 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (830573a1-b372-473d-b0ef-118dbafd8f4b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:49.013818 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (830573a1-b372-473d-b0ef-118dbafd8f4b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:49.019785 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8194edc4-48e1-4cb7-8070-eb2aebfe0cec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:49.025250 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8194edc4-48e1-4cb7-8070-eb2aebfe0cec) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd8ac5cbc-2d8c-4e85-a955-ecde664d63a4', 'volume_name': 'volume-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:49.030877 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4e87c896-7aa8-45ad-831d-8652fe893ae4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:49.035422 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4e87c896-7aa8-45ad-831d-8652fe893ae4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:49.040916 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (885c8951-1f69-41a0-9b7a-715fadfa2938) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:49.042613 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Volume d8ac5cbc-2d8c-4e85-a955-ecde664d63a4: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4', 'volume_size': 1} Sep 27 10:17:49.043250 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] creating volume 'volume-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:17:49.045070 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:49.209638 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:17:49.210473 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:17:49.221669 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (885c8951-1f69-41a0-9b7a-715fadfa2938) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:49.248570 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (46060440-4e1a-4d69-ab36-44e4f8441eb1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:49.282891 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Volume volume-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4 (d8ac5cbc-2d8c-4e85-a955-ecde664d63a4): created successfully Sep 27 10:17:49.301050 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (46060440-4e1a-4d69-ab36-44e4f8441eb1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:49.321481 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Flow 'volume_create_manager' (0b3a3339-9e6c-4a19-bbf6-bf0a0eeddd09) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:49.356287 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-2ce7a620-f177-4d07-b388-433953a9c41f tempest-AttachVolumeTestJSON-1476802274 None] Created volume successfully. Sep 27 10:17:52.115600 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 req-405b5b06-30a8-4db1-aa96-e69103867978 tempest-AttachVolumeTestJSON-1476802274 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:17:53.573865 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 req-405b5b06-30a8-4db1-aa96-e69103867978 tempest-AttachVolumeTestJSON-1476802274 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.462s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:17:53.574690 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 req-405b5b06-30a8-4db1-aa96-e69103867978 tempest-AttachVolumeTestJSON-1476802274 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'd8ac5cbc-2d8c-4e85-a955-ecde664d63a4', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:17:53.659708 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 req-405b5b06-30a8-4db1-aa96-e69103867978 tempest-AttachVolumeTestJSON-1476802274 None] Connection info returned from driver {'name': 'volumes/volume-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'd8ac5cbc-2d8c-4e85-a955-ecde664d63a4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'c9d830d9-e096-4ecb-83dc-01b76bcbe469'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:17:53.715939 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=63,cinder:UPDATE=29,cinder:INSERT=7,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:53.805004 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 req-405b5b06-30a8-4db1-aa96-e69103867978 tempest-AttachVolumeTestJSON-1476802274 None] attachment_update completed successfully. Sep 27 10:17:54.683506 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:17:54.729820 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Flow 'volume_create_manager' (a0fe23bc-f2eb-4368-9767-5c941990c4b2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:54.736392 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4d5aa6a7-6639-4e5f-be8e-3aa10c173d06) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:54.774480 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4d5aa6a7-6639-4e5f-be8e-3aa10c173d06) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:17:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesImageMetadata-Volume-487111629',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=54af652d-9ebf-41f0-9f3e-4ea6ec9522f9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fb57a25dc6b944f18a5eec9f6ac089ca',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:17:55Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:17:55Z,use_quota=True,user_id='807fffcc89334a54b52cddf575180929',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:54.784139 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fe9decff-df83-4291-ab55-bd4d38aa66b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:54.788415 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fe9decff-df83-4291-ab55-bd4d38aa66b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:54.806159 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e5fb1d45-cce1-43ab-887d-7f9d145f2291) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:55.022374 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e5fb1d45-cce1-43ab-887d-7f9d145f2291) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '54af652d-9ebf-41f0-9f3e-4ea6ec9522f9', 'volume_name': 'volume-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:55.034013 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e756b267-1612-4694-9bb4-9529bd6ded9d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:55.038094 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e756b267-1612-4694-9bb4-9529bd6ded9d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:55.063802 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (28eb6874-e6ad-4948-b6f7-a3f84549a6e6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:55.068041 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Volume 54af652d-9ebf-41f0-9f3e-4ea6ec9522f9: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:17:55.072627 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Cloning 54af652d-9ebf-41f0-9f3e-4ea6ec9522f9 from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:17:55.073681 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:17:55.084239 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:17:55.099802 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 17, 55, 88938)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:17:55.100412 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:17:55.101111 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:17:55.111769 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.028s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:17:55.113597 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:17:55.124910 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 17, 55, 121968)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:17:55.125663 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:17:55.126246 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:17:55.142194 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:55.250964 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] creating snapshot='volume-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:17:55.802685 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9.clone_snap' to 'volume-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:17:55.919803 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] Acquiring lock "cinder-4b2f9c8c-667b-4bff-8590-c93510f8acf3-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:17:56.005353 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:17:56.007232 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:17:56.007839 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:17:56.038496 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:17:56.039222 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Marking volume 54af652d-9ebf-41f0-9f3e-4ea6ec9522f9 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:17:56.051221 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] Lock "cinder-4b2f9c8c-667b-4bff-8590-c93510f8acf3-delete_volume" acquired by "delete_volume" :: waited 0.134s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:17:56.078182 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 54af652d-9ebf-41f0-9f3e-4ea6ec9522f9. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:17:56.079274 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Creating volume glance metadata for volume 54af652d-9ebf-41f0-9f3e-4ea6ec9522f9 backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:17:56.164236 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:56.264520 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:17:56.266890 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] Volume volume-4b2f9c8c-667b-4bff-8590-c93510f8acf3 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:17:56.298272 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] deleting rbd volume volume-4b2f9c8c-667b-4bff-8590-c93510f8acf3 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:17:56.444825 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (28eb6874-e6ad-4948-b6f7-a3f84549a6e6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:56.464442 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ecf4b5d7-55c9-4c8e-a7ca-241943614381) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:17:56.522870 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Volume volume-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9 (54af652d-9ebf-41f0-9f3e-4ea6ec9522f9): created successfully Sep 27 10:17:56.538526 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ecf4b5d7-55c9-4c8e-a7ca-241943614381) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:17:56.563996 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Flow 'volume_create_manager' (a0fe23bc-f2eb-4368-9767-5c941990c4b2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:17:56.580793 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=71,cinder:UPDATE=23,cinder:DELETE=1,cinder:INSERT=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:56.619324 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c0eab231-8812-41de-b196-4ea129e051d7 tempest-VolumesImageMetadata-1043173891 None] Created volume successfully. Sep 27 10:17:56.886959 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] Created reservations ['5a21ed3b-57a2-459b-8356-0a78ddd8e24c', 'b11d8443-0464-4ae9-b86e-ebae635a9602', '37eb8edf-944e-41f7-b804-c228d924ff02', '4882aa40-dc4a-4971-b187-21380234c58b'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:17:56.988394 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:57.067561 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:57.161831 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:17:57.181004 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] Acquiring lock "cinder-8943dda0-13e7-42e3-bdeb-f9cf11b10058-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:17:57.185059 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] Deleted volume successfully. Sep 27 10:17:57.205056 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] Lock "cinder-8943dda0-13e7-42e3-bdeb-f9cf11b10058-delete_volume" acquired by "delete_volume" :: waited 0.024s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:17:57.220939 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5c0d1120-d33a-4892-a777-f370c01fd07f tempest-TaggedBootDevicesTest-1222598264 None] Lock "cinder-4b2f9c8c-667b-4bff-8590-c93510f8acf3-delete_volume" released by "delete_volume" :: held 1.170s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:17:57.290937 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:57.423738 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:17:57.424084 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] Volume volume-8943dda0-13e7-42e3-bdeb-f9cf11b10058 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:17:57.438734 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] deleting rbd volume volume-8943dda0-13e7-42e3-bdeb-f9cf11b10058 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:17:57.465309 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea req-d662b996-977c-496d-9c9d-d0f5018da5ac tempest-AttachVolumeTestJSON-1476802274 None] Terminate volume connection completed successfully. Sep 27 10:17:57.465742 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea req-d662b996-977c-496d-9c9d-d0f5018da5ac tempest-AttachVolumeTestJSON-1476802274 None] Deleting attachment 3c1af1a2-c21d-4da4-9546-999d9bce70fc. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:17:57.526639 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] Acquiring lock "cinder-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:17:57.544788 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] Lock "cinder-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:17:57.630528 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:57.789072 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:17:57.813687 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] deleting rbd volume volume-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:17:58.056472 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] Created reservations ['6740c597-a54d-4c6c-bd96-0e389607245e', 'aa1ac95e-53bf-489a-b4e1-5baba0d9dcf0', '46e963fa-469c-4f3e-a5a8-ff15da53dcd6', 'c366d1ee-0dca-44d2-9e10-a1eb642ecefe'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:17:58.168215 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:58.259175 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:17:58.302456 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] deleting parent snapshot volume-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:17:58.938585 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:58.993484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:17:58.997220 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] Acquiring lock "cinder-cceacac7-f31d-4e36-b764-2231c93d64eb-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:17:59.021450 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] Deleted volume successfully. Sep 27 10:17:59.048471 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] Lock "cinder-cceacac7-f31d-4e36-b764-2231c93d64eb-delete_volume" acquired by "delete_volume" :: waited 0.051s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:17:59.067166 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=62,cinder:UPDATE=19,cinder:INSERT=17,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:59.081255 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-832a5ed3-6e96-456d-a7b1-28b26a679f1f tempest-TaggedBootDevicesTest-1222598264 None] Lock "cinder-8943dda0-13e7-42e3-bdeb-f9cf11b10058-delete_volume" released by "delete_volume" :: held 1.877s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:17:59.084699 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] Created reservations ['c3300a28-3279-4144-914a-a5012068f146', '180e7718-a84f-49e0-95b7-3558bb8d03fb', 'baebc0a5-d7ec-4314-82b1-05fc967c0c27', 'f2318d41-6fc7-4b88-b386-eb03c0033774'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:17:59.134615 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:59.277928 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:17:59.290826 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] deleting rbd volume volume-cceacac7-f31d-4e36-b764-2231c93d64eb {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:17:59.333296 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:59.387455 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:17:59.458302 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:17:59.499638 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] Deleted volume successfully. Sep 27 10:17:59.552898 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-5c45dfb8-dedf-4342-b447-471698568987 req-7f277ae1-a6bb-45fc-826d-aeee3dcb1eab tempest-AttachVolumeTestJSON-1476802274 None] Terminate volume connection completed successfully. Sep 27 10:17:59.564973 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5c45dfb8-dedf-4342-b447-471698568987 req-7f277ae1-a6bb-45fc-826d-aeee3dcb1eab tempest-AttachVolumeTestJSON-1476802274 None] Deleting attachment c9d830d9-e096-4ecb-83dc-01b76bcbe469. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:17:59.579754 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6dc555b9-144e-4d52-99c7-01d356ee0549 tempest-VolumesImageMetadata-1043173891 None] Lock "cinder-54af652d-9ebf-41f0-9f3e-4ea6ec9522f9-delete_volume" released by "delete_volume" :: held 2.036s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:17:59.762398 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:17:59.801369 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] deleting parent snapshot volume-cceacac7-f31d-4e36-b764-2231c93d64eb.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:18:01.093134 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] Created reservations ['8cdabb48-b5f4-401b-ae66-efc4cc40d202', '4c5d7093-3fd6-4c54-b0a5-b7fa815d1dc1', 'c0327288-3681-4dd0-b38a-b32d071ed48e', '4e5ec536-85be-4f4e-9d44-b06e2b605c7d'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:18:01.270155 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:01.342966 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:01.392026 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:18:01.399147 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] Deleted volume successfully. Sep 27 10:18:01.404178 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=49,cinder:INSERT=7,cinder:UPDATE=42,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:01.433571 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-58d6d79b-5429-41fb-b7a0-cf8ac8ec9c47 tempest-TaggedBootDevicesTest-1222598264 None] Lock "cinder-cceacac7-f31d-4e36-b764-2231c93d64eb-delete_volume" released by "delete_volume" :: held 2.385s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:18:06.799829 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:18:06.837519 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Flow 'volume_create_manager' (af52da3b-383f-4f9e-bdbb-a6e325dd3846) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:18:06.852778 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9c03ef25-42be-464e-8cd6-0c2f630f78b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:06.873966 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9c03ef25-42be-464e-8cd6-0c2f630f78b9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:18:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV355Test-Volume-439058187',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d42aa3f0-8b91-4db3-a1c6-ab235a44034d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e70b2dec4993431c865eb7d2f28e32bf',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:18:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:18:07Z,use_quota=True,user_id='110cdeadcd4842f3b9a3e284c05d955a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:06.879924 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b4356202-75c2-41a3-a33b-9e9b8da29639) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:06.884479 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b4356202-75c2-41a3-a33b-9e9b8da29639) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:06.890501 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bc4f3202-3c78-41a6-b00f-1287a5595f66) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:06.895361 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bc4f3202-3c78-41a6-b00f-1287a5595f66) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd42aa3f0-8b91-4db3-a1c6-ab235a44034d', 'volume_name': 'volume-d42aa3f0-8b91-4db3-a1c6-ab235a44034d', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:06.900513 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cde8b30d-1653-4b65-b32e-1953807f3897) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:06.905251 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cde8b30d-1653-4b65-b32e-1953807f3897) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:06.910145 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d286e36b-6298-469e-9847-be6cba761793) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:06.911477 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Volume d42aa3f0-8b91-4db3-a1c6-ab235a44034d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d42aa3f0-8b91-4db3-a1c6-ab235a44034d', 'volume_size': 1} Sep 27 10:18:06.912555 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] creating volume 'volume-d42aa3f0-8b91-4db3-a1c6-ab235a44034d' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:18:06.914182 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:07.081590 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:18:07.082212 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:18:07.086375 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d286e36b-6298-469e-9847-be6cba761793) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d42aa3f0-8b91-4db3-a1c6-ab235a44034d', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:07.096353 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1ac78823-8464-4de7-9879-db5292a8022e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:07.114209 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Volume volume-d42aa3f0-8b91-4db3-a1c6-ab235a44034d (d42aa3f0-8b91-4db3-a1c6-ab235a44034d): created successfully Sep 27 10:18:07.126868 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1ac78823-8464-4de7-9879-db5292a8022e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:07.166429 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Flow 'volume_create_manager' (af52da3b-383f-4f9e-bdbb-a6e325dd3846) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:18:07.209987 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0bb722af-3456-4ff8-bfa1-9a93e775cc1d tempest-VolumesTransfersV355Test-1997832806 None] Created volume successfully. Sep 27 10:18:09.099582 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-201990fb-1399-4715-8192-8711fc3ec4d6 tempest-VolumesTransfersV355Test-258984861 None] Transfer volume completed successfully. Sep 27 10:18:09.853516 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] Acquiring lock "cinder-d42aa3f0-8b91-4db3-a1c6-ab235a44034d-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:18:09.868737 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] Lock "cinder-d42aa3f0-8b91-4db3-a1c6-ab235a44034d-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:18:09.915421 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:09.993388 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:18:09.993804 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] Volume volume-d42aa3f0-8b91-4db3-a1c6-ab235a44034d is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:18:10.001547 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] deleting rbd volume volume-d42aa3f0-8b91-4db3-a1c6-ab235a44034d {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:18:10.315516 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] Created reservations ['7072d3c0-03b6-4d12-8521-a2bb9af65d2b', '09408839-8e33-4b14-be82-a0ccf2f8bf4d', 'ad2883f5-218d-41f6-8bc1-5e3773bc0ee5', 'ad06efbc-ff5b-422f-a8e5-3c133bf66001'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:18:10.387331 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:10.447678 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:10.521787 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:18:10.527127 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] Deleted volume successfully. Sep 27 10:18:10.561683 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-ca338dd5-ea17-4a9b-ba8e-7249663ca973 tempest-VolumesTransfersV355Test-1373577637 None] Lock "cinder-d42aa3f0-8b91-4db3-a1c6-ab235a44034d-delete_volume" released by "delete_volume" :: held 0.694s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:18:10.787573 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] Acquiring lock "cinder-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:18:10.801309 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] Lock "cinder-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:18:10.844247 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:10.924923 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:18:10.925347 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] Volume volume-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:18:10.931457 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] deleting rbd volume volume-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:18:11.284317 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] Created reservations ['721e6b4e-3c1e-44bd-82a0-44141ac187df', '09a6136e-a323-42e2-bd70-58335f0982a2', 'a3e2ce69-1834-447f-885d-0aa50275f393', '134dda96-ebeb-4f55-9f6d-36f6caf4e514'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:18:11.366732 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:11.434928 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:11.505951 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:18:11.571413 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] Deleted volume successfully. Sep 27 10:18:11.605472 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:18:11.632591 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Flow 'volume_create_manager' (bc138b49-7bb7-474e-a519-8dc7b99e838a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:18:11.636331 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e0938ca7-a1c7-4b73-981b-615a5028480e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:11.640736 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-fa65af71-0a60-46b6-957e-3368ecd2385f tempest-AttachVolumeTestJSON-1476802274 None] Lock "cinder-d8ac5cbc-2d8c-4e85-a955-ecde664d63a4-delete_volume" released by "delete_volume" :: held 0.840s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:18:11.658040 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e0938ca7-a1c7-4b73-981b-615a5028480e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:18:11Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV355Test-Volume-1376345297',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=db6b528c-5201-4976-9484-cf976d5555de,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e70b2dec4993431c865eb7d2f28e32bf',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:18:11Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:18:11Z,use_quota=True,user_id='110cdeadcd4842f3b9a3e284c05d955a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:11.664594 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (03d45aeb-f831-471e-b60f-22187e81a084) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:11.670827 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (03d45aeb-f831-471e-b60f-22187e81a084) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:11.677264 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (001b294e-8e3d-4c29-8aa9-2209ed7e77a5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:11.681376 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (001b294e-8e3d-4c29-8aa9-2209ed7e77a5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'db6b528c-5201-4976-9484-cf976d5555de', 'volume_name': 'volume-db6b528c-5201-4976-9484-cf976d5555de', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:11.686496 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c97fba5e-fa6d-461a-9b5f-397e8c5e40b4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:11.690266 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c97fba5e-fa6d-461a-9b5f-397e8c5e40b4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:11.695764 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6239fdda-5b43-445f-9f48-e5c3fab804d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:11.697205 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Volume db6b528c-5201-4976-9484-cf976d5555de: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-db6b528c-5201-4976-9484-cf976d5555de', 'volume_size': 1} Sep 27 10:18:11.697762 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] creating volume 'volume-db6b528c-5201-4976-9484-cf976d5555de' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:18:11.698932 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:11.820758 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:18:11.822167 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:18:11.827364 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6239fdda-5b43-445f-9f48-e5c3fab804d7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-db6b528c-5201-4976-9484-cf976d5555de', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:11.842251 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9ad8a110-2da3-464a-b1f3-50b86cd7308e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:18:11.871556 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Volume volume-db6b528c-5201-4976-9484-cf976d5555de (db6b528c-5201-4976-9484-cf976d5555de): created successfully Sep 27 10:18:11.882639 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9ad8a110-2da3-464a-b1f3-50b86cd7308e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:18:11.894168 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Flow 'volume_create_manager' (bc138b49-7bb7-474e-a519-8dc7b99e838a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:18:11.915178 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e42821b4-db25-408a-b19a-a7274bbdbe15 tempest-VolumesTransfersV355Test-1997832806 None] Created volume successfully. Sep 27 10:18:11.972521 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] Acquiring lock "cinder-3bc57786-1606-4e4f-81f1-868696186ba0-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:18:11.998781 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] Lock "cinder-3bc57786-1606-4e4f-81f1-868696186ba0-delete_volume" acquired by "delete_volume" :: waited 0.026s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:18:12.045912 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:12.132349 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:18:12.132791 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] Volume volume-3bc57786-1606-4e4f-81f1-868696186ba0 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:18:12.138294 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] deleting rbd volume volume-3bc57786-1606-4e4f-81f1-868696186ba0 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:18:12.485090 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=54,cinder:INSERT=6,cinder:UPDATE=37,cinder:DELETE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:12.508814 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] Created reservations ['2461a6ac-cc79-49c1-b30a-05674e13acc2', '3cdd5921-e8fc-4f9b-8cf3-532fcf4a2f2a', 'b9de5613-02d3-47cd-8aa9-1654e2a042df', 'c50cb600-e238-4ea3-8527-a71d4b42ce08'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:18:12.597407 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:12.653813 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:12.714157 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:18:12.719678 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] Deleted volume successfully. Sep 27 10:18:12.741130 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0ecb0df9-8706-4397-b0d4-ff2af06eab26 tempest-AttachVolumeTestJSON-1476802274 None] Lock "cinder-3bc57786-1606-4e4f-81f1-868696186ba0-delete_volume" released by "delete_volume" :: held 0.742s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:18:12.957579 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] Acquiring lock "cinder-db6b528c-5201-4976-9484-cf976d5555de-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:18:12.984160 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] Lock "cinder-db6b528c-5201-4976-9484-cf976d5555de-delete_volume" acquired by "delete_volume" :: waited 0.028s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:18:13.043158 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:13.129425 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:18:13.129859 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] Volume volume-db6b528c-5201-4976-9484-cf976d5555de is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:18:13.147578 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] deleting rbd volume volume-db6b528c-5201-4976-9484-cf976d5555de {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:18:13.366487 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] Acquiring lock "cinder-e5af107c-5b71-461c-a3e1-deb10425a0f9-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:18:13.380612 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] Lock "cinder-e5af107c-5b71-461c-a3e1-deb10425a0f9-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:18:13.461458 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:13.571060 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:18:13.571679 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] Volume volume-e5af107c-5b71-461c-a3e1-deb10425a0f9 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:18:13.603789 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] deleting rbd volume volume-e5af107c-5b71-461c-a3e1-deb10425a0f9 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:18:13.708861 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] Created reservations ['caa4f9dd-e6da-4444-ab62-9fe4ac38cd04', '6b8944a8-6cb2-4270-a9af-274d545fc456', 'd73acf00-01ed-46d7-8d38-02d42312a90e', '054b464f-aebe-49e8-9f16-998bc0471e14'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:18:13.754073 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=61,cinder:UPDATE=28,cinder:DELETE=2,cinder:INSERT=9 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:13.832367 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:13.899480 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:13.985684 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:18:13.991555 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] Deleted volume successfully. Sep 27 10:18:14.029388 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-604f547e-3f23-4b69-8723-a5adbce948e4 tempest-VolumesTransfersV355Test-1373577637 None] Lock "cinder-db6b528c-5201-4976-9484-cf976d5555de-delete_volume" released by "delete_volume" :: held 1.045s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:18:14.061069 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] Created reservations ['cf9f32f3-6109-44ed-ad0f-fa288b4c5dee', 'b3309c22-8c0b-494a-84ab-84d83cceba4c', '10fe1a38-b6bf-47f6-8bf4-6743e2ddafcb', 'c0107dbf-c3ec-4c2f-9f92-1e3a53f26d61'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:18:14.150818 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:14.250582 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:14.337444 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:18:14.353204 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] Deleted volume successfully. Sep 27 10:18:14.384054 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5c14d750-990b-48db-8601-90a883483ca0 tempest-AttachVolumeTestJSON-1476802274 None] Lock "cinder-e5af107c-5b71-461c-a3e1-deb10425a0f9-delete_volume" released by "delete_volume" :: held 1.003s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:18:26.115612 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=9,cinder:UPDATE=8 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:29.765870 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:18:29.766614 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:29.832954 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:18:29.948601 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:18:46.113974 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=32,cinder:UPDATE=24,cinder:DELETE=3,cinder:INSERT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:06.118512 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:16.119578 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:29.774350 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:19:29.776050 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:29.909014 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:29.995020 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:19:31.482878 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-8f2e2b13-6726-47bc-8a09-af60d9ccb024 tempest-GroupsV314Test-336464008 None] Group tempest-GroupsV314Test-Group-1413137060: creating Sep 27 10:19:31.568259 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-8f2e2b13-6726-47bc-8a09-af60d9ccb024 tempest-GroupsV314Test-336464008 None] Group tempest-GroupsV314Test-Group-1413137060: created successfully Sep 27 10:19:31.589812 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-8f2e2b13-6726-47bc-8a09-af60d9ccb024 tempest-GroupsV314Test-336464008 None] Create group completed successfully. Sep 27 10:19:33.420414 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:19:33.464133 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Flow 'volume_create_manager' (31eec4aa-2d07-4b0f-a995-2cd39ff0a27f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:19:33.470733 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4307724e-cef4-4fa1-924e-4e88c0d0614e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:33.494648 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4307724e-cef4-4fa1-924e-4e88c0d0614e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:19:33Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsV314Test-Volume-982362804',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=52f99172-3ca6-4928-92db-bc643f690bca,host='ncdv-master-1@ceph#ceph',id=9c0d1a5c-805d-4d7e-80f1-8136f37398e1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0ff624ba07b54ec985a94783d529cdef',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:19:33Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:19:33Z,use_quota=True,user_id='3d42fa166bb14724a1ddffed4a6c6231',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c360b931-3878-4ead-8b00-0c468f8f3f39),volume_type_id=c360b931-3878-4ead-8b00-0c468f8f3f39)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:33.500478 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f9ef39aa-bc14-44bd-9b8a-eff6b772337a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:33.508457 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f9ef39aa-bc14-44bd-9b8a-eff6b772337a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:33.515311 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b295ca87-dbf8-4fa9-8522-83877ba246b5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:33.519828 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b295ca87-dbf8-4fa9-8522-83877ba246b5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '9c0d1a5c-805d-4d7e-80f1-8136f37398e1', 'volume_name': 'volume-9c0d1a5c-805d-4d7e-80f1-8136f37398e1', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:33.529923 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (33c86ffd-98d9-424a-84e5-43b8619a8c0d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:33.535395 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (33c86ffd-98d9-424a-84e5-43b8619a8c0d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:33.540979 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f3c95529-3827-4c24-aff4-816a65c3fad2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:33.542628 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Volume 9c0d1a5c-805d-4d7e-80f1-8136f37398e1: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-9c0d1a5c-805d-4d7e-80f1-8136f37398e1', 'volume_size': 1} Sep 27 10:19:33.543400 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] creating volume 'volume-9c0d1a5c-805d-4d7e-80f1-8136f37398e1' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:19:33.544463 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:33.678562 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:19:33.678940 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:19:33.682635 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f3c95529-3827-4c24-aff4-816a65c3fad2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9c0d1a5c-805d-4d7e-80f1-8136f37398e1', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:33.689974 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d79f7972-c0e2-4d0c-9209-7e49c1136812) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:33.711184 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Volume volume-9c0d1a5c-805d-4d7e-80f1-8136f37398e1 (9c0d1a5c-805d-4d7e-80f1-8136f37398e1): created successfully Sep 27 10:19:33.716010 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d79f7972-c0e2-4d0c-9209-7e49c1136812) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:33.724081 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Flow 'volume_create_manager' (31eec4aa-2d07-4b0f-a995-2cd39ff0a27f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:19:33.744773 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-a6517e39-f6d5-4590-94f1-65c3df5136c9 tempest-GroupsV314Test-336464008 None] Created volume successfully. Sep 27 10:19:35.802424 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-01c91a87-e81e-4a62-a88a-c4ff976bc7c9 tempest-GroupsV314Test-336464008 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:35.941796 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-01c91a87-e81e-4a62-a88a-c4ff976bc7c9 tempest-GroupsV314Test-336464008 None] creating snapshot='volume-4ee9dd0a-9e9f-4736-981c-110fb721be2f.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:19:37.027299 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-01c91a87-e81e-4a62-a88a-c4ff976bc7c9 tempest-GroupsV314Test-336464008 None] cloning 'volume-9c0d1a5c-805d-4d7e-80f1-8136f37398e1@volume-4ee9dd0a-9e9f-4736-981c-110fb721be2f.clone_snap' to 'volume-4ee9dd0a-9e9f-4736-981c-110fb721be2f' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:19:37.040198 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=19,cinder:UPDATE=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:37.221646 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-01c91a87-e81e-4a62-a88a-c4ff976bc7c9 tempest-GroupsV314Test-336464008 None] Volume volume-9c0d1a5c-805d-4d7e-80f1-8136f37398e1 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:19:37.236170 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-01c91a87-e81e-4a62-a88a-c4ff976bc7c9 tempest-GroupsV314Test-336464008 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:19:37.236899 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-01c91a87-e81e-4a62-a88a-c4ff976bc7c9 tempest-GroupsV314Test-336464008 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:19:37.280253 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-01c91a87-e81e-4a62-a88a-c4ff976bc7c9 tempest-GroupsV314Test-336464008 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:19:37.383278 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-01c91a87-e81e-4a62-a88a-c4ff976bc7c9 tempest-GroupsV314Test-336464008 None] Create group from source-group-52f99172-3ca6-4928-92db-bc643f690bca completed successfully. Sep 27 10:19:39.285728 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:39.445101 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:19:39.463223 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] deleting rbd volume volume-4ee9dd0a-9e9f-4736-981c-110fb721be2f {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:19:39.897786 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:19:39.932640 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] Volume volume-9c0d1a5c-805d-4d7e-80f1-8136f37398e1 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:19:39.933278 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] deleting parent snapshot volume-4ee9dd0a-9e9f-4736-981c-110fb721be2f.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:19:40.327892 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] Created reservations ['e3f08872-e745-4c2e-841e-6ae66cf3087e'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:19:40.376176 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] Deprecated: Default quota for resource: volumes_tempest-GroupsV314Test-volume-type-1397889475 is set by the default quota flag: quota_volumes_tempest-GroupsV314Test-volume-type-1397889475, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:19:40.377222 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsV314Test-volume-type-1397889475 is set by the default quota flag: quota_gigabytes_tempest-GroupsV314Test-volume-type-1397889475, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:19:40.411006 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] Created reservations ['e3467029-74ba-41b2-8a19-51a0648a7362', 'b655b40d-207c-4732-9a73-3b2b25a949e0', '7f3fdc29-ad01-4970-819b-3d8e528d6ffb', '467abdf6-2c64-407d-8218-f77f2b6d6c1f'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:19:40.560718 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:40.627596 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:40.695193 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:19:40.706581 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e14c8799-2fb4-4e48-92f8-05a83712fbea tempest-GroupsV314Test-336464008 None] Delete group completed successfully. Sep 27 10:19:42.118494 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-81dcf488-8108-43a4-bb63-43e44b5af61c tempest-GroupsV314Test-336464008 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:42.237165 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-81dcf488-8108-43a4-bb63-43e44b5af61c tempest-GroupsV314Test-336464008 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:19:42.237587 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-81dcf488-8108-43a4-bb63-43e44b5af61c tempest-GroupsV314Test-336464008 None] Volume volume-9c0d1a5c-805d-4d7e-80f1-8136f37398e1 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:19:42.250573 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-81dcf488-8108-43a4-bb63-43e44b5af61c tempest-GroupsV314Test-336464008 None] deleting rbd volume volume-9c0d1a5c-805d-4d7e-80f1-8136f37398e1 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:19:42.737603 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-81dcf488-8108-43a4-bb63-43e44b5af61c tempest-GroupsV314Test-336464008 None] Created reservations ['d832fa50-a931-4fee-87b0-fbc1ec80918e'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:19:42.820570 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-81dcf488-8108-43a4-bb63-43e44b5af61c tempest-GroupsV314Test-336464008 None] Created reservations ['12b6bd22-cc75-4082-80e9-177af2a51440', '4f7a7245-5932-42a6-80f5-7d8224b0d720', '85e11481-b558-4865-9ec1-1cbad08eea33', '753922fd-01e7-4611-92fc-1d227f3cf033'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:19:42.990520 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-81dcf488-8108-43a4-bb63-43e44b5af61c tempest-GroupsV314Test-336464008 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:43.083662 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-81dcf488-8108-43a4-bb63-43e44b5af61c tempest-GroupsV314Test-336464008 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:43.159867 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-81dcf488-8108-43a4-bb63-43e44b5af61c tempest-GroupsV314Test-336464008 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:19:43.184290 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-81dcf488-8108-43a4-bb63-43e44b5af61c tempest-GroupsV314Test-336464008 None] Delete group completed successfully. Sep 27 10:19:47.031405 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=60,cinder:UPDATE=29,cinder:DELETE=1,cinder:INSERT=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:51.672683 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:19:51.790310 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Flow 'volume_create_manager' (c57f76b8-28e6-44e2-9331-952668f0725a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:19:51.803902 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0c416380-756d-486d-948a-ff1ff4ba0384) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:51.852503 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0c416380-756d-486d-948a-ff1ff4ba0384) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:19:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotsActionsTest-Volume-660976526',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=046b2ce4-c7cd-4df6-a9ec-6741c596f90c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='1d576d7a3ba44e14ba171a3c8ee6a18a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:19:52Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:19:52Z,use_quota=True,user_id='46a22f58246647e9a990b3865fa5c4c1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:51.860008 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e37bc1d9-63b1-4fc2-b7b9-c776b04fe1e8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:51.865752 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e37bc1d9-63b1-4fc2-b7b9-c776b04fe1e8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:51.882429 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (77e10d1e-455e-44c3-af5e-b3c72efbee3b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:51.891028 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (77e10d1e-455e-44c3-af5e-b3c72efbee3b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '046b2ce4-c7cd-4df6-a9ec-6741c596f90c', 'volume_name': 'volume-046b2ce4-c7cd-4df6-a9ec-6741c596f90c', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:51.896876 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (079c68db-ab70-4332-9d6a-7a7b86fad9d2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:51.901100 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (079c68db-ab70-4332-9d6a-7a7b86fad9d2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:51.907261 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6dbf4147-9cb1-44e7-a813-2fd29c72e663) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:51.909283 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Volume 046b2ce4-c7cd-4df6-a9ec-6741c596f90c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-046b2ce4-c7cd-4df6-a9ec-6741c596f90c', 'volume_size': 1} Sep 27 10:19:51.909902 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] creating volume 'volume-046b2ce4-c7cd-4df6-a9ec-6741c596f90c' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:19:51.911511 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:52.081469 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:19:52.081849 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:19:52.085578 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6dbf4147-9cb1-44e7-a813-2fd29c72e663) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-046b2ce4-c7cd-4df6-a9ec-6741c596f90c', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:52.090082 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (43191754-ff60-4bb4-8f33-5df546c4947d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:19:52.115577 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Volume volume-046b2ce4-c7cd-4df6-a9ec-6741c596f90c (046b2ce4-c7cd-4df6-a9ec-6741c596f90c): created successfully Sep 27 10:19:52.121712 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (43191754-ff60-4bb4-8f33-5df546c4947d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:19:52.137550 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Flow 'volume_create_manager' (c57f76b8-28e6-44e2-9331-952668f0725a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:19:52.163484 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-83987894-9c8c-4e2b-8c76-078ab4d54a8e tempest-SnapshotsActionsTest-1766773568 None] Created volume successfully. Sep 27 10:19:53.228387 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ce9bcc5a-05fb-46d7-ae91-850caa13c98a tempest-SnapshotsActionsTest-1766773568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:54.406924 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ce9bcc5a-05fb-46d7-ae91-850caa13c98a tempest-SnapshotsActionsTest-1766773568 None] Create snapshot completed successfully Sep 27 10:19:56.198762 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f1ae95f4-bb89-42ef-a11b-91f4ce7a1bd8 tempest-SnapshotsActionsTest-1766773568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:57.469458 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-f1ae95f4-bb89-42ef-a11b-91f4ce7a1bd8 tempest-SnapshotsActionsTest-1766773568 None] Create snapshot completed successfully Sep 27 10:19:58.425723 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-24164a63-28d3-4a48-8737-99cea5f9da1e tempest-SnapshotsActionsTest-892573409 None] Acquiring lock "cinder-9dde7fe3-b09c-4ecd-b08d-24cbdb7953fc-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:19:58.440984 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-24164a63-28d3-4a48-8737-99cea5f9da1e tempest-SnapshotsActionsTest-892573409 None] Lock "cinder-9dde7fe3-b09c-4ecd-b08d-24cbdb7953fc-delete_snapshot" acquired by "delete_snapshot" :: waited 0.014s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:19:58.442743 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-24164a63-28d3-4a48-8737-99cea5f9da1e tempest-SnapshotsActionsTest-892573409 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:19:59.603904 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=66,cinder:UPDATE=28,cinder:INSERT=5,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:59.691083 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-24164a63-28d3-4a48-8737-99cea5f9da1e tempest-SnapshotsActionsTest-892573409 None] Created reservations ['b18554d8-0e4f-4524-a8e4-f170d737ac91', '8118886e-e233-4e65-821c-e4a884e1ba09', '47acd60a-ee62-4f7d-88a7-2de97d21c364', '8630ca81-497f-497d-b4e2-93d5c0be564c'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:19:59.804372 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-24164a63-28d3-4a48-8737-99cea5f9da1e tempest-SnapshotsActionsTest-892573409 None] Delete snapshot completed successfully. Sep 27 10:19:59.822768 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-24164a63-28d3-4a48-8737-99cea5f9da1e tempest-SnapshotsActionsTest-892573409 None] Lock "cinder-9dde7fe3-b09c-4ecd-b08d-24cbdb7953fc-delete_snapshot" released by "delete_snapshot" :: held 1.383s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:20:01.027877 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6af64fff-2fb1-4ea1-a394-5ab3fc842503 tempest-SnapshotsActionsTest-1766773568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:01.690379 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6af64fff-2fb1-4ea1-a394-5ab3fc842503 tempest-SnapshotsActionsTest-1766773568 None] Create snapshot completed successfully Sep 27 10:20:02.312460 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-99673fe0-5e1e-4d26-a98c-daf89f175a3a tempest-SnapshotsActionsTest-892573409 None] Acquiring lock "cinder-d648c927-62da-4985-ab43-b016d5e3af37-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:20:02.324623 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-99673fe0-5e1e-4d26-a98c-daf89f175a3a tempest-SnapshotsActionsTest-892573409 None] Lock "cinder-d648c927-62da-4985-ab43-b016d5e3af37-delete_snapshot" acquired by "delete_snapshot" :: waited 0.012s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:20:02.328716 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-99673fe0-5e1e-4d26-a98c-daf89f175a3a tempest-SnapshotsActionsTest-892573409 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:03.847823 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-99673fe0-5e1e-4d26-a98c-daf89f175a3a tempest-SnapshotsActionsTest-892573409 None] Created reservations ['9acd2601-2c24-4d57-a928-8836ddce434d', '3f270216-2f27-448d-95de-2d3106c6b614', 'ffb075dd-6d1e-4af6-8335-e5d0ddc68ecc', '4ff7337a-c196-4fa4-a67f-ca559174ed54'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:20:03.946631 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-99673fe0-5e1e-4d26-a98c-daf89f175a3a tempest-SnapshotsActionsTest-892573409 None] Delete snapshot completed successfully. Sep 27 10:20:03.952269 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-99673fe0-5e1e-4d26-a98c-daf89f175a3a tempest-SnapshotsActionsTest-892573409 None] Lock "cinder-d648c927-62da-4985-ab43-b016d5e3af37-delete_snapshot" released by "delete_snapshot" :: held 1.628s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:20:05.133483 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-61934700-d6f7-4885-89d8-82db49543e26 tempest-SnapshotsActionsTest-1766773568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:05.888779 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-61934700-d6f7-4885-89d8-82db49543e26 tempest-SnapshotsActionsTest-1766773568 None] Create snapshot completed successfully Sep 27 10:20:06.410742 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d22091ba-db3f-45cc-80ee-85e84865e7e9 tempest-SnapshotsActionsTest-892573409 None] Acquiring lock "cinder-d823daca-6c4f-48c8-86a2-59fb60c641ea-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:20:06.426645 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d22091ba-db3f-45cc-80ee-85e84865e7e9 tempest-SnapshotsActionsTest-892573409 None] Lock "cinder-d823daca-6c4f-48c8-86a2-59fb60c641ea-delete_snapshot" acquired by "delete_snapshot" :: waited 0.016s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:20:06.428218 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d22091ba-db3f-45cc-80ee-85e84865e7e9 tempest-SnapshotsActionsTest-892573409 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:06.852410 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-d22091ba-db3f-45cc-80ee-85e84865e7e9 tempest-SnapshotsActionsTest-892573409 None] Created reservations ['b1d1c935-76da-4f56-9d87-e6375c3c60c0', '8c32c3e4-305c-4b4a-a9f2-43c94509d068', '585f06ad-642b-4616-bac0-13294bf0eccc', '2e63f164-d07f-49a1-8814-77051a7424da'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:20:06.898550 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d22091ba-db3f-45cc-80ee-85e84865e7e9 tempest-SnapshotsActionsTest-892573409 None] Delete snapshot completed successfully. Sep 27 10:20:06.904479 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d22091ba-db3f-45cc-80ee-85e84865e7e9 tempest-SnapshotsActionsTest-892573409 None] Lock "cinder-d823daca-6c4f-48c8-86a2-59fb60c641ea-delete_snapshot" released by "delete_snapshot" :: held 0.478s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:20:07.964567 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5b62b74c-e5a1-461a-a6f9-dc6385a3d10f tempest-SnapshotsActionsTest-1766773568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:09.029504 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5b62b74c-e5a1-461a-a6f9-dc6385a3d10f tempest-SnapshotsActionsTest-1766773568 None] Create snapshot completed successfully Sep 27 10:20:10.274922 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d9241e16-9933-42d6-b647-2f82e4e41738 tempest-SnapshotsActionsTest-892573409 None] Acquiring lock "cinder-2e011c02-66d8-4299-9fc4-e8b62114532e-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:20:10.308084 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d9241e16-9933-42d6-b647-2f82e4e41738 tempest-SnapshotsActionsTest-892573409 None] Lock "cinder-2e011c02-66d8-4299-9fc4-e8b62114532e-delete_snapshot" acquired by "delete_snapshot" :: waited 0.033s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:20:10.310902 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d9241e16-9933-42d6-b647-2f82e4e41738 tempest-SnapshotsActionsTest-892573409 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:11.074187 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=62,cinder:DELETE=4,cinder:INSERT=10,cinder:UPDATE=24 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:11.087070 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-d9241e16-9933-42d6-b647-2f82e4e41738 tempest-SnapshotsActionsTest-892573409 None] Created reservations ['6772e794-7eb8-4b8e-922e-eb566006aef9', 'a80560d4-209a-4237-9d4f-caa4d9abe856', '4f6871e3-0b34-4917-ae89-3392a8a07b87', 'ab8b110a-3071-475b-ba64-de538415a961'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:20:11.152797 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d9241e16-9933-42d6-b647-2f82e4e41738 tempest-SnapshotsActionsTest-892573409 None] Delete snapshot completed successfully. Sep 27 10:20:11.159825 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d9241e16-9933-42d6-b647-2f82e4e41738 tempest-SnapshotsActionsTest-892573409 None] Lock "cinder-2e011c02-66d8-4299-9fc4-e8b62114532e-delete_snapshot" released by "delete_snapshot" :: held 0.852s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:20:12.290997 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-739c4126-402d-4e3a-9e9b-06fc4f910162 tempest-SnapshotsActionsTest-1766773568 None] Acquiring lock "cinder-7d6d07e2-fb0c-46e7-99df-f2223d766de0-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:20:12.306845 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-739c4126-402d-4e3a-9e9b-06fc4f910162 tempest-SnapshotsActionsTest-1766773568 None] Lock "cinder-7d6d07e2-fb0c-46e7-99df-f2223d766de0-delete_snapshot" acquired by "delete_snapshot" :: waited 0.016s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:20:12.312624 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-739c4126-402d-4e3a-9e9b-06fc4f910162 tempest-SnapshotsActionsTest-1766773568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:13.226317 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-739c4126-402d-4e3a-9e9b-06fc4f910162 tempest-SnapshotsActionsTest-1766773568 None] Created reservations ['132a4808-f68b-425b-bb7b-4c6c010ebc01', 'e54f3bdb-3161-454a-a255-c5aae6cb125f', '50dba804-2026-433e-b039-550fe65b740e', 'bb61a1c9-9da3-4812-98fb-6cb48ab71f78'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:20:13.312586 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-739c4126-402d-4e3a-9e9b-06fc4f910162 tempest-SnapshotsActionsTest-1766773568 None] Delete snapshot completed successfully. Sep 27 10:20:13.316847 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-739c4126-402d-4e3a-9e9b-06fc4f910162 tempest-SnapshotsActionsTest-1766773568 None] Lock "cinder-7d6d07e2-fb0c-46e7-99df-f2223d766de0-delete_snapshot" released by "delete_snapshot" :: held 1.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:20:13.500014 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] Acquiring lock "cinder-046b2ce4-c7cd-4df6-a9ec-6741c596f90c-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:20:13.528349 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] Lock "cinder-046b2ce4-c7cd-4df6-a9ec-6741c596f90c-delete_volume" acquired by "delete_volume" :: waited 0.028s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:20:13.618705 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:13.717184 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:20:13.718029 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] Volume volume-046b2ce4-c7cd-4df6-a9ec-6741c596f90c is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:20:13.727887 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] deleting rbd volume volume-046b2ce4-c7cd-4df6-a9ec-6741c596f90c {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:20:14.254819 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] Created reservations ['080d4488-934a-4a69-8d7d-5d39ac200e3f', 'f4338bda-260f-4ab4-9083-a1084458e466', '5b663474-8198-4f71-ba51-d153f8c979ab', 'e4c92655-9fc7-4bc9-84b7-b8b507da2ed5'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:20:14.321205 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=60,cinder:UPDATE=30,cinder:INSERT=9,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:14.335633 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:14.404804 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:14.459755 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:20:14.464114 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] Deleted volume successfully. Sep 27 10:20:14.481337 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-23677999-55ed-4aa3-9be8-aefaa731eb05 tempest-SnapshotsActionsTest-1766773568 None] Lock "cinder-046b2ce4-c7cd-4df6-a9ec-6741c596f90c-delete_volume" released by "delete_volume" :: held 0.953s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:20:24.469668 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:29.772803 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:20:29.774082 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:29.837407 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:29.943826 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:20:31.130638 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:20:31.180366 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Flow 'volume_create_manager' (bde4723f-014c-4f9b-b057-ea7b4164d5d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:20:31.187451 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (07fcff3e-f310-405a-8973-06fcb76a303e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:31.210752 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (07fcff3e-f310-405a-8973-06fcb76a303e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:20:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTestJSON-Volume-67048335',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=61fd1543-5e89-47fc-a9e5-641aa5252e59,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='4f80f3a9dcfe43c6868d300dd1495253',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:20:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:20:31Z,use_quota=True,user_id='4c6f77ddc6a245db9d9b236145c7647f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:31.216887 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (71097e74-0854-4810-98ae-b0fdff785d58) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:31.222479 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (71097e74-0854-4810-98ae-b0fdff785d58) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:31.228926 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (16b587a7-6e3a-4456-806b-3ea211656974) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:31.234318 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (16b587a7-6e3a-4456-806b-3ea211656974) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '61fd1543-5e89-47fc-a9e5-641aa5252e59', 'volume_name': 'volume-61fd1543-5e89-47fc-a9e5-641aa5252e59', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:31.241393 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5b7514f4-dab8-4ffe-962e-cabddd9d1670) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:31.247112 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5b7514f4-dab8-4ffe-962e-cabddd9d1670) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:31.253577 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aaf635d5-c962-492c-89c8-ec07ea9683d5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:31.255302 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Volume 61fd1543-5e89-47fc-a9e5-641aa5252e59: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-61fd1543-5e89-47fc-a9e5-641aa5252e59', 'volume_size': 1} Sep 27 10:20:31.256149 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] creating volume 'volume-61fd1543-5e89-47fc-a9e5-641aa5252e59' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:20:31.257298 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:31.374570 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:20:31.375001 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:20:31.380473 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aaf635d5-c962-492c-89c8-ec07ea9683d5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-61fd1543-5e89-47fc-a9e5-641aa5252e59', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:31.387146 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bf3af449-63c3-4feb-8d7a-4a36c380e063) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:31.399426 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Volume volume-61fd1543-5e89-47fc-a9e5-641aa5252e59 (61fd1543-5e89-47fc-a9e5-641aa5252e59): created successfully Sep 27 10:20:31.404361 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bf3af449-63c3-4feb-8d7a-4a36c380e063) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:31.411580 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Flow 'volume_create_manager' (bde4723f-014c-4f9b-b057-ea7b4164d5d8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:20:31.425630 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-75423c69-ad4e-4114-ad7f-6a1d951fad49 req-4cbf1e56-3416-4aa4-b6a8-0de6eee0509f tempest-VolumesGetTestJSON-1505950546 None] Created volume successfully. Sep 27 10:20:32.663152 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] Acquiring lock "cinder-61fd1543-5e89-47fc-a9e5-641aa5252e59-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:20:32.682393 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] Lock "cinder-61fd1543-5e89-47fc-a9e5-641aa5252e59-delete_volume" acquired by "delete_volume" :: waited 0.020s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:20:32.752738 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:32.821601 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:20:32.822372 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] Volume volume-61fd1543-5e89-47fc-a9e5-641aa5252e59 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:20:32.835617 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] deleting rbd volume volume-61fd1543-5e89-47fc-a9e5-641aa5252e59 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:20:33.177545 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] Created reservations ['1439347a-62e2-4dde-9736-5fc41faf7950', 'd46d4fd3-788f-4dab-b311-da9419e49673', '3cab4b0a-511a-4ee5-bf13-8ce1b9fbaeee', '67d63c0f-6b54-4e9d-b598-54a29a1d8d32'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:20:33.268297 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:33.359830 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:33.454309 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:20:33.468814 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] Deleted volume successfully. Sep 27 10:20:33.501121 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 req-c1321513-0023-4e5a-b730-4706610d6e78 tempest-VolumesGetTestJSON-1505950546 None] Lock "cinder-61fd1543-5e89-47fc-a9e5-641aa5252e59-delete_volume" released by "delete_volume" :: held 0.818s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:20:45.885134 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=52,cinder:UPDATE=36,cinder:INSERT=8 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:45.902857 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:20:45.941366 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Flow 'volume_create_manager' (92ee0cdb-4a20-4322-8a61-306a89c6e21a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:20:45.951548 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (606cb202-e152-4f02-a2aa-8ef5b59642f6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:46.046031 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (606cb202-e152-4f02-a2aa-8ef5b59642f6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:20:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-1118884150',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=05adfdbe-11b6-4e2e-9ace-45f0a99dba1d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b590d2de576e41aab42f79e4daef9c63',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:20:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:20:46Z,use_quota=True,user_id='74878b715fc94b23bb139cb30d8d20c0',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:46.059798 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ef2f248d-2b35-46ef-b17a-4369c4948e14) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:46.073255 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ef2f248d-2b35-46ef-b17a-4369c4948e14) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:46.088214 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f7ef66a0-c863-4314-845e-cc9734745acd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:46.098586 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f7ef66a0-c863-4314-845e-cc9734745acd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '05adfdbe-11b6-4e2e-9ace-45f0a99dba1d', 'volume_name': 'volume-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:46.105863 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2a5a1b82-9078-42d4-ade0-728d8676b687) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:46.123081 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2a5a1b82-9078-42d4-ade0-728d8676b687) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:46.142678 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (786041c1-7918-46e6-a6c1-0c9c57e7fb07) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:46.144365 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Volume 05adfdbe-11b6-4e2e-9ace-45f0a99dba1d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d', 'volume_size': 1} Sep 27 10:20:46.146677 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] creating volume 'volume-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:20:46.151438 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:46.306879 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:20:46.307572 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:20:46.313081 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (786041c1-7918-46e6-a6c1-0c9c57e7fb07) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:46.322629 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (07dbc257-3c9b-49f3-812e-73cc49c8215e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:46.347285 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Volume volume-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d (05adfdbe-11b6-4e2e-9ace-45f0a99dba1d): created successfully Sep 27 10:20:46.352648 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (07dbc257-3c9b-49f3-812e-73cc49c8215e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:46.365185 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Flow 'volume_create_manager' (92ee0cdb-4a20-4322-8a61-306a89c6e21a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:20:46.383074 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-dc7e0c07-0b15-4281-8117-4daaa54fb30b tempest-AttachVolumeShelveTestJSON-1912240963 None] Created volume successfully. Sep 27 10:20:57.031585 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=27,cinder:UPDATE=13,cinder:DELETE=2,cinder:INSERT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:58.901080 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:20:58.933045 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Flow 'volume_create_manager' (2d27d47b-64e8-496d-baff-67af8345f97a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:20:58.958806 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fb362ba3-1464-4205-bcef-ada058bb8372) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:58.983125 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fb362ba3-1464-4205-bcef-ada058bb8372) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:20:58Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Volume-1577102871',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d1ae8a98-2bfa-4160-8013-a149f5f71fb8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='140e3033364a498c9752cae22f3108a6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:20:59Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:20:59Z,use_quota=True,user_id='1be0da58001e4b71976eaf46bd246b2d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:58.989442 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7a4e9e3d-4470-4281-8198-eae6585420a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:58.994320 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7a4e9e3d-4470-4281-8198-eae6585420a8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:59.002130 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (03a51854-02f6-4702-9453-b77daa90faff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:59.006862 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (03a51854-02f6-4702-9453-b77daa90faff) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd1ae8a98-2bfa-4160-8013-a149f5f71fb8', 'volume_name': 'volume-d1ae8a98-2bfa-4160-8013-a149f5f71fb8', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:59.012351 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bdc67dfd-f6a1-41d6-badc-54efc8819f25) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:59.016641 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bdc67dfd-f6a1-41d6-badc-54efc8819f25) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:59.021538 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c07bb591-2e9e-4ce2-a66c-cd089bce172a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:59.023205 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Volume d1ae8a98-2bfa-4160-8013-a149f5f71fb8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d1ae8a98-2bfa-4160-8013-a149f5f71fb8', 'volume_size': 1} Sep 27 10:20:59.023956 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] creating volume 'volume-d1ae8a98-2bfa-4160-8013-a149f5f71fb8' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:20:59.024890 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:20:59.143179 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:20:59.143851 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:20:59.150021 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c07bb591-2e9e-4ce2-a66c-cd089bce172a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d1ae8a98-2bfa-4160-8013-a149f5f71fb8', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:59.156622 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1206794b-395e-4d65-8ca9-5cec36f02723) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:20:59.172607 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Volume volume-d1ae8a98-2bfa-4160-8013-a149f5f71fb8 (d1ae8a98-2bfa-4160-8013-a149f5f71fb8): created successfully Sep 27 10:20:59.181565 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1206794b-395e-4d65-8ca9-5cec36f02723) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:20:59.188686 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Flow 'volume_create_manager' (2d27d47b-64e8-496d-baff-67af8345f97a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:20:59.201132 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-066bc11c-9434-4a56-b1ba-e7c21498e957 tempest-VolumesBackupsAdminTest-253385919 None] Created volume successfully. Sep 27 10:21:00.576842 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-128c9124-12c3-4650-80d2-8131ff4d27ce tempest-VolumesBackupsAdminTest-253385919 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-27T10:21:00Z,data_timestamp=2022-09-27T10:21:00Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-241964801',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=7aa8d318-ede7-4d5f-996b-fad47dd104ea,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='140e3033364a498c9752cae22f3108a6',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='1be0da58001e4b71976eaf46bd246b2d',volume_id=d1ae8a98-2bfa-4160-8013-a149f5f71fb8) Sep 27 10:21:00.603385 ncdv-master-1 cinder-volume[138473]: INFO cinder.rpc [None req-128c9124-12c3-4650-80d2-8131ff4d27ce tempest-VolumesBackupsAdminTest-253385919 None] Automatically selected cinder-backup objects version 1.39 as minimum service version. Sep 27 10:21:00.634183 ncdv-master-1 cinder-volume[138473]: INFO cinder.rpc [None req-128c9124-12c3-4650-80d2-8131ff4d27ce tempest-VolumesBackupsAdminTest-253385919 None] Automatically selected cinder-backup RPC version 2.3 as minimum service version. Sep 27 10:21:00.635358 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.backup.rpcapi [None req-128c9124-12c3-4650-80d2-8131ff4d27ce tempest-VolumesBackupsAdminTest-253385919 None] continue_backup in rpcapi backup_id 7aa8d318-ede7-4d5f-996b-fad47dd104ea {{(pid=138473) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 27 10:21:11.442545 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-128c9124-12c3-4650-80d2-8131ff4d27ce tempest-VolumesBackupsAdminTest-253385919 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:21:13.391138 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-128c9124-12c3-4650-80d2-8131ff4d27ce tempest-VolumesBackupsAdminTest-253385919 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.948s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:21:13.394849 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-128c9124-12c3-4650-80d2-8131ff4d27ce tempest-VolumesBackupsAdminTest-253385919 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-d1ae8a98-2bfa-4160-8013-a149f5f71fb8', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'd1ae8a98-2bfa-4160-8013-a149f5f71fb8', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:21:13.432990 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-128c9124-12c3-4650-80d2-8131ff4d27ce tempest-VolumesBackupsAdminTest-253385919 None] Initialize volume connection completed successfully. Sep 27 10:21:17.039185 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:DELETE=2,cinder:SELECT=20,cinder:UPDATE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:18.119634 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-128c9124-12c3-4650-80d2-8131ff4d27ce tempest-VolumesBackupsAdminTest-253385919 None] Terminate volume connection completed successfully. Sep 27 10:21:18.172199 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-128c9124-12c3-4650-80d2-8131ff4d27ce tempest-VolumesBackupsAdminTest-253385919 None] Remove volume export completed successfully. Sep 27 10:21:21.741875 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:21:21.813648 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Flow 'volume_create_manager' (710228e8-c0df-4bcc-9aa7-6ef48edf5515) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:21:21.828469 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (625bfa35-b66d-4f5a-be0b-179e5bc15313) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:21.880834 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (625bfa35-b66d-4f5a-be0b-179e5bc15313) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:21:21Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_7aa8d318-ede7-4d5f-996b-fad47dd104ea',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d6c0aaf7-3597-47e9-8caa-33dc05dcc035,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='140e3033364a498c9752cae22f3108a6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:21:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:21:22Z,use_quota=True,user_id='1be0da58001e4b71976eaf46bd246b2d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:21.888222 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a4fdfa9b-ac42-45c3-9e50-3e25bc366e7f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:21.894163 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a4fdfa9b-ac42-45c3-9e50-3e25bc366e7f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:21.900175 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b36b3cf9-60e1-4de0-a962-1538596b4f30) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:21.904640 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b36b3cf9-60e1-4de0-a962-1538596b4f30) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd6c0aaf7-3597-47e9-8caa-33dc05dcc035', 'volume_name': 'volume-d6c0aaf7-3597-47e9-8caa-33dc05dcc035', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:21.912211 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a0d622d-b12f-45b0-ac3d-deff0a421dbf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:21.917977 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a0d622d-b12f-45b0-ac3d-deff0a421dbf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:21.927155 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4830c8e0-5623-4796-8dba-f4c15925a4d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:21.932102 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Volume d6c0aaf7-3597-47e9-8caa-33dc05dcc035: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d6c0aaf7-3597-47e9-8caa-33dc05dcc035', 'volume_size': 1} Sep 27 10:21:21.932989 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] creating volume 'volume-d6c0aaf7-3597-47e9-8caa-33dc05dcc035' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:21:21.935078 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:22.122890 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:21:22.123793 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:21:22.129344 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4830c8e0-5623-4796-8dba-f4c15925a4d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d6c0aaf7-3597-47e9-8caa-33dc05dcc035', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:22.141784 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a5821e61-61e8-423d-b480-b8f9444f74af) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:22.171856 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Volume volume-d6c0aaf7-3597-47e9-8caa-33dc05dcc035 (d6c0aaf7-3597-47e9-8caa-33dc05dcc035): created successfully Sep 27 10:21:22.176178 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a5821e61-61e8-423d-b480-b8f9444f74af) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:22.188025 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Flow 'volume_create_manager' (710228e8-c0df-4bcc-9aa7-6ef48edf5515) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:21:22.210730 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Created volume successfully. Sep 27 10:21:22.838745 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:21:24.700729 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.862s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:21:24.702356 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-d6c0aaf7-3597-47e9-8caa-33dc05dcc035', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'd6c0aaf7-3597-47e9-8caa-33dc05dcc035', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:21:24.746010 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Initialize volume connection completed successfully. Sep 27 10:21:26.692013 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Terminate volume connection completed successfully. Sep 27 10:21:26.725318 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e5f94e8b-538d-47dd-aae0-ca19c43ff3c1 tempest-VolumesBackupsAdminTest-253385919 None] Remove volume export completed successfully. Sep 27 10:21:28.290376 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-eb540061-21b8-4d73-a038-d9375f7d9139 tempest-VolumesBackupsAdminTest-253385919 None] Acquiring lock "cinder-d6c0aaf7-3597-47e9-8caa-33dc05dcc035-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:21:28.317157 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-eb540061-21b8-4d73-a038-d9375f7d9139 tempest-VolumesBackupsAdminTest-253385919 None] Lock "cinder-d6c0aaf7-3597-47e9-8caa-33dc05dcc035-delete_volume" acquired by "delete_volume" :: waited 0.025s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:21:28.328777 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=21,cinder:UPDATE=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:28.431613 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-eb540061-21b8-4d73-a038-d9375f7d9139 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:29.324639 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-eb540061-21b8-4d73-a038-d9375f7d9139 tempest-VolumesBackupsAdminTest-253385919 None] Volume volume-d6c0aaf7-3597-47e9-8caa-33dc05dcc035 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:21:29.363630 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-eb540061-21b8-4d73-a038-d9375f7d9139 tempest-VolumesBackupsAdminTest-253385919 None] deleting rbd volume volume-d6c0aaf7-3597-47e9-8caa-33dc05dcc035 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:21:29.769173 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:21:29.770175 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:29.882203 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:29.985115 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:21:30.113967 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-eb540061-21b8-4d73-a038-d9375f7d9139 tempest-VolumesBackupsAdminTest-253385919 None] Created reservations ['beaf5098-4d99-404e-b08b-dd978813dd4b', '9c7e063d-0d13-43a7-a567-1bb5187649e3', '3e34aa0f-ebe3-4eb0-980f-8349ec8e36f9', '40a541e9-662d-44c6-855e-474e1d2e4078'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:21:30.231668 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-eb540061-21b8-4d73-a038-d9375f7d9139 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:30.343209 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-eb540061-21b8-4d73-a038-d9375f7d9139 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:30.433281 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-eb540061-21b8-4d73-a038-d9375f7d9139 tempest-VolumesBackupsAdminTest-253385919 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:21:30.450591 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-eb540061-21b8-4d73-a038-d9375f7d9139 tempest-VolumesBackupsAdminTest-253385919 None] Deleted volume successfully. Sep 27 10:21:30.483324 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-eb540061-21b8-4d73-a038-d9375f7d9139 tempest-VolumesBackupsAdminTest-253385919 None] Lock "cinder-d6c0aaf7-3597-47e9-8caa-33dc05dcc035-delete_volume" released by "delete_volume" :: held 2.167s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:21:32.456760 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:21:32.487827 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Flow 'volume_create_manager' (d4b2b282-c10c-49a8-a999-1cd4a554bc9f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:21:32.495320 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (033792a6-ab25-49c1-8c90-0ec38dc312b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:32.516548 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (033792a6-ab25-49c1-8c90-0ec38dc312b1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:21:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Volume-2014180246',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a5156cae-5268-47be-90fe-43fe5eda8721,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='140e3033364a498c9752cae22f3108a6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:21:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:21:32Z,use_quota=True,user_id='1be0da58001e4b71976eaf46bd246b2d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:32.533164 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (814ff8f7-3e62-4173-ad52-733212096d46) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:32.533164 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (814ff8f7-3e62-4173-ad52-733212096d46) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:32.537059 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (963bb2e1-af7d-423d-b7c0-64cdfd1fb57e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:32.542255 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (963bb2e1-af7d-423d-b7c0-64cdfd1fb57e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a5156cae-5268-47be-90fe-43fe5eda8721', 'volume_name': 'volume-a5156cae-5268-47be-90fe-43fe5eda8721', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:32.547572 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1b39aeac-6e9d-49dc-a5b7-25e476237ab3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:32.553512 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1b39aeac-6e9d-49dc-a5b7-25e476237ab3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:32.558439 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d901afcf-87d2-4547-a105-7d7fdfada630) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:32.559902 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Volume a5156cae-5268-47be-90fe-43fe5eda8721: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a5156cae-5268-47be-90fe-43fe5eda8721', 'volume_size': 1} Sep 27 10:21:32.560475 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] creating volume 'volume-a5156cae-5268-47be-90fe-43fe5eda8721' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:21:32.561519 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:32.691286 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:21:32.691570 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:21:32.696181 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d901afcf-87d2-4547-a105-7d7fdfada630) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a5156cae-5268-47be-90fe-43fe5eda8721', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:32.702851 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c267480e-720c-464a-85cf-ba4762a53351) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:32.715440 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Volume volume-a5156cae-5268-47be-90fe-43fe5eda8721 (a5156cae-5268-47be-90fe-43fe5eda8721): created successfully Sep 27 10:21:32.721634 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c267480e-720c-464a-85cf-ba4762a53351) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:32.736764 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Flow 'volume_create_manager' (d4b2b282-c10c-49a8-a999-1cd4a554bc9f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:21:32.751084 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6f0ee7db-81ba-47b3-9d6b-4269167c1a64 tempest-VolumesBackupsAdminTest-253385919 None] Created volume successfully. Sep 27 10:21:33.915082 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e895e7ba-ba42-43d4-97a6-d8b9b0653f46 tempest-VolumesBackupsAdminTest-253385919 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-27T10:21:33Z,data_timestamp=2022-09-27T10:21:33Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-2043764666',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=377d8650-c009-46aa-b23c-2de708cf6cc6,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='140e3033364a498c9752cae22f3108a6',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='1be0da58001e4b71976eaf46bd246b2d',volume_id=a5156cae-5268-47be-90fe-43fe5eda8721) Sep 27 10:21:33.921547 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.backup.rpcapi [None req-e895e7ba-ba42-43d4-97a6-d8b9b0653f46 tempest-VolumesBackupsAdminTest-253385919 None] continue_backup in rpcapi backup_id 377d8650-c009-46aa-b23c-2de708cf6cc6 {{(pid=138473) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 27 10:21:34.134547 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-e895e7ba-ba42-43d4-97a6-d8b9b0653f46 tempest-VolumesBackupsAdminTest-253385919 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:21:35.762364 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-e895e7ba-ba42-43d4-97a6-d8b9b0653f46 tempest-VolumesBackupsAdminTest-253385919 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.627s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:21:35.764215 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e895e7ba-ba42-43d4-97a6-d8b9b0653f46 tempest-VolumesBackupsAdminTest-253385919 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-a5156cae-5268-47be-90fe-43fe5eda8721', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'a5156cae-5268-47be-90fe-43fe5eda8721', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:21:35.793909 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e895e7ba-ba42-43d4-97a6-d8b9b0653f46 tempest-VolumesBackupsAdminTest-253385919 None] Initialize volume connection completed successfully. Sep 27 10:21:37.543359 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e895e7ba-ba42-43d4-97a6-d8b9b0653f46 tempest-VolumesBackupsAdminTest-253385919 None] Terminate volume connection completed successfully. Sep 27 10:21:37.606765 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e895e7ba-ba42-43d4-97a6-d8b9b0653f46 tempest-VolumesBackupsAdminTest-253385919 None] Remove volume export completed successfully. Sep 27 10:21:39.990569 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-012d7a88-7071-4cf2-95a6-6d66547d80a9 req-f1ec8c30-1787-4b9a-875b-0b91fc440a48 tempest-AttachVolumeShelveTestJSON-1912240963 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:21:41.126768 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-012d7a88-7071-4cf2-95a6-6d66547d80a9 req-f1ec8c30-1787-4b9a-875b-0b91fc440a48 tempest-AttachVolumeShelveTestJSON-1912240963 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.136s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:21:41.127637 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-012d7a88-7071-4cf2-95a6-6d66547d80a9 req-f1ec8c30-1787-4b9a-875b-0b91fc440a48 tempest-AttachVolumeShelveTestJSON-1912240963 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '05adfdbe-11b6-4e2e-9ace-45f0a99dba1d', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:21:41.186903 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-012d7a88-7071-4cf2-95a6-6d66547d80a9 req-f1ec8c30-1787-4b9a-875b-0b91fc440a48 tempest-AttachVolumeShelveTestJSON-1912240963 None] Connection info returned from driver {'name': 'volumes/volume-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '05adfdbe-11b6-4e2e-9ace-45f0a99dba1d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'd41b920c-93e8-433a-8ece-d27e2d50c153'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:21:41.294859 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-012d7a88-7071-4cf2-95a6-6d66547d80a9 req-f1ec8c30-1787-4b9a-875b-0b91fc440a48 tempest-AttachVolumeShelveTestJSON-1912240963 None] attachment_update completed successfully. Sep 27 10:21:41.660684 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c78067bf-21da-4fd0-97e8-b69dee032ac5 tempest-VolumesBackupsAdminTest-253385919 None] Acquiring lock "cinder-a5156cae-5268-47be-90fe-43fe5eda8721-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:21:41.685777 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c78067bf-21da-4fd0-97e8-b69dee032ac5 tempest-VolumesBackupsAdminTest-253385919 None] Lock "cinder-a5156cae-5268-47be-90fe-43fe5eda8721-delete_volume" acquired by "delete_volume" :: waited 0.023s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:21:41.792931 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c78067bf-21da-4fd0-97e8-b69dee032ac5 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:42.651736 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c78067bf-21da-4fd0-97e8-b69dee032ac5 tempest-VolumesBackupsAdminTest-253385919 None] Volume volume-a5156cae-5268-47be-90fe-43fe5eda8721 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:21:42.682675 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c78067bf-21da-4fd0-97e8-b69dee032ac5 tempest-VolumesBackupsAdminTest-253385919 None] deleting rbd volume volume-a5156cae-5268-47be-90fe-43fe5eda8721 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:21:43.012982 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-c78067bf-21da-4fd0-97e8-b69dee032ac5 tempest-VolumesBackupsAdminTest-253385919 None] Created reservations ['7f85df30-f118-4fc8-b640-dead699042f3', '4c652b86-1b7c-4520-afc2-b6398ac96edd', '24d3e6a7-7cfd-4b1f-9015-6ee67892faf9', '05eec67b-773c-4b0b-945b-5e54887864b3'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:21:43.126828 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c78067bf-21da-4fd0-97e8-b69dee032ac5 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:43.217779 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c78067bf-21da-4fd0-97e8-b69dee032ac5 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:43.307170 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-c78067bf-21da-4fd0-97e8-b69dee032ac5 tempest-VolumesBackupsAdminTest-253385919 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:21:43.317438 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c78067bf-21da-4fd0-97e8-b69dee032ac5 tempest-VolumesBackupsAdminTest-253385919 None] Deleted volume successfully. Sep 27 10:21:43.341390 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c78067bf-21da-4fd0-97e8-b69dee032ac5 tempest-VolumesBackupsAdminTest-253385919 None] Lock "cinder-a5156cae-5268-47be-90fe-43fe5eda8721-delete_volume" released by "delete_volume" :: held 1.658s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:21:43.928703 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-af8825f2-cd08-4048-9d69-ebe79f8fe9b1 tempest-VolumesBackupsAdminTest-253385919 None] Acquiring lock "cinder-d1ae8a98-2bfa-4160-8013-a149f5f71fb8-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:21:43.945092 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-af8825f2-cd08-4048-9d69-ebe79f8fe9b1 tempest-VolumesBackupsAdminTest-253385919 None] Lock "cinder-d1ae8a98-2bfa-4160-8013-a149f5f71fb8-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:21:43.980225 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=65,cinder:UPDATE=27,cinder:DELETE=4,cinder:INSERT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:44.012634 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-af8825f2-cd08-4048-9d69-ebe79f8fe9b1 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:44.727801 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-af8825f2-cd08-4048-9d69-ebe79f8fe9b1 tempest-VolumesBackupsAdminTest-253385919 None] Volume volume-d1ae8a98-2bfa-4160-8013-a149f5f71fb8 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:21:44.772343 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-af8825f2-cd08-4048-9d69-ebe79f8fe9b1 tempest-VolumesBackupsAdminTest-253385919 None] deleting rbd volume volume-d1ae8a98-2bfa-4160-8013-a149f5f71fb8 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:21:45.293500 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-af8825f2-cd08-4048-9d69-ebe79f8fe9b1 tempest-VolumesBackupsAdminTest-253385919 None] Created reservations ['06d2b9e2-e175-4f3a-bca8-653a58cfede3', 'ed0a14a8-359c-4117-9592-f1ac31977d46', 'a5ee013e-e5b5-4ea6-afc7-8532dc3a8244', '03b5512c-99cc-40af-bd1c-ddfffc8b36b9'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:21:45.461384 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-af8825f2-cd08-4048-9d69-ebe79f8fe9b1 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:45.555210 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-af8825f2-cd08-4048-9d69-ebe79f8fe9b1 tempest-VolumesBackupsAdminTest-253385919 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:45.626569 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-af8825f2-cd08-4048-9d69-ebe79f8fe9b1 tempest-VolumesBackupsAdminTest-253385919 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:21:45.633623 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-af8825f2-cd08-4048-9d69-ebe79f8fe9b1 tempest-VolumesBackupsAdminTest-253385919 None] Deleted volume successfully. Sep 27 10:21:45.667527 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-af8825f2-cd08-4048-9d69-ebe79f8fe9b1 tempest-VolumesBackupsAdminTest-253385919 None] Lock "cinder-d1ae8a98-2bfa-4160-8013-a149f5f71fb8-delete_volume" released by "delete_volume" :: held 1.723s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:21:57.037673 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=9,cinder:UPDATE=9 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:58.568398 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=60,cinder:INSERT=8,cinder:UPDATE=23,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:58.626472 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:21:58.677260 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Flow 'volume_create_manager' (cd877864-d630-4875-b6f4-92b3b79199ec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:21:58.688281 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5c307ed0-da1e-4edc-85ab-696dedee6f4c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:58.759921 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5c307ed0-da1e-4edc-85ab-696dedee6f4c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:21:58Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesDeleteCascade-Volume-479522946',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e8e28128-e0c5-4f4a-80f2-ee68453209ee,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b8fae88fc8f8420eb7f5b18520f4a822',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:21:59Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:21:59Z,use_quota=True,user_id='0d6b172b40364a4aba6f55e0cb24ea4e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:58.771630 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (35cb10ce-7a37-493b-afa5-10d096052a5a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:58.776167 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (35cb10ce-7a37-493b-afa5-10d096052a5a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:58.781650 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (12a915d7-d90d-4e81-bf0a-58b03ba3e2d6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:58.787210 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (12a915d7-d90d-4e81-bf0a-58b03ba3e2d6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e8e28128-e0c5-4f4a-80f2-ee68453209ee', 'volume_name': 'volume-e8e28128-e0c5-4f4a-80f2-ee68453209ee', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:58.799838 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (698dffc4-1bb5-472f-8746-3c3b3b24aa57) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:58.809169 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (698dffc4-1bb5-472f-8746-3c3b3b24aa57) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:58.815126 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2dc58327-6cac-4e86-a15e-bec6bc7bbee2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:58.816430 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Volume e8e28128-e0c5-4f4a-80f2-ee68453209ee: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e8e28128-e0c5-4f4a-80f2-ee68453209ee', 'volume_size': 1} Sep 27 10:21:58.817187 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] creating volume 'volume-e8e28128-e0c5-4f4a-80f2-ee68453209ee' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:21:58.818207 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:21:58.954778 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:21:58.955152 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:21:58.959347 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2dc58327-6cac-4e86-a15e-bec6bc7bbee2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e8e28128-e0c5-4f4a-80f2-ee68453209ee', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:58.963710 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9c43d768-8730-4011-9812-2e3aec36091b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:21:58.998805 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Volume volume-e8e28128-e0c5-4f4a-80f2-ee68453209ee (e8e28128-e0c5-4f4a-80f2-ee68453209ee): created successfully Sep 27 10:21:59.010967 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9c43d768-8730-4011-9812-2e3aec36091b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:21:59.027633 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Flow 'volume_create_manager' (cd877864-d630-4875-b6f4-92b3b79199ec) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:21:59.043343 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c824b695-7d25-418a-9366-70f4ce56ea5c tempest-VolumesDeleteCascade-1826627454 None] Created volume successfully. Sep 27 10:22:00.050553 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ce655c2-7671-412b-a6fb-5650c8596d6d tempest-VolumesDeleteCascade-1826627454 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:00.429299 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5ce655c2-7671-412b-a6fb-5650c8596d6d tempest-VolumesDeleteCascade-1826627454 None] Create snapshot completed successfully Sep 27 10:22:01.426447 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-653e6380-555b-429e-a9d9-e61f1a131b0f tempest-VolumesDeleteCascade-1826627454 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:02.602236 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-653e6380-555b-429e-a9d9-e61f1a131b0f tempest-VolumesDeleteCascade-1826627454 None] Create snapshot completed successfully Sep 27 10:22:03.598681 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Acquiring lock "cinder-e8e28128-e0c5-4f4a-80f2-ee68453209ee-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:22:03.616360 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Lock "cinder-e8e28128-e0c5-4f4a-80f2-ee68453209ee-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:22:03.676325 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Performing cascade delete. {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/manager.py:1008}} Sep 27 10:22:03.695938 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Acquiring lock "cinder-28cf0dcc-19e4-4759-8005-a6867229ed39-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:22:03.749829 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Lock "cinder-28cf0dcc-19e4-4759-8005-a6867229ed39-delete_snapshot" acquired by "delete_snapshot" :: waited 0.053s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:22:03.783081 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:04.672373 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Created reservations ['78acf122-0bf8-48a6-b758-2a076742315c', '6a61763d-1655-41c0-bf8b-92e038ec950c', '4e7bc561-42d3-4069-960e-b68317be9cdb', '19479447-8fef-4587-95f0-99c61bde4677'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:22:04.756214 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Delete snapshot completed successfully. Sep 27 10:22:04.762555 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Lock "cinder-28cf0dcc-19e4-4759-8005-a6867229ed39-delete_snapshot" released by "delete_snapshot" :: held 1.014s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:22:04.763744 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Acquiring lock "cinder-c9f4ec9c-7494-4525-b5c0-5e6fcd338166-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:22:04.780019 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Lock "cinder-c9f4ec9c-7494-4525-b5c0-5e6fcd338166-delete_snapshot" acquired by "delete_snapshot" :: waited 0.016s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:22:04.809029 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:05.654729 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Created reservations ['d367d465-561f-4c08-a7a4-57d01eda4dd3', 'b7ed7aa6-65bb-47cf-9fbf-d03f5417fde3', '41352938-eb72-4290-aa80-44e2009450a7', '47ab161a-a369-4675-b070-a21b707a972a'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:22:05.742111 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Delete snapshot completed successfully. Sep 27 10:22:05.749363 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Lock "cinder-c9f4ec9c-7494-4525-b5c0-5e6fcd338166-delete_snapshot" released by "delete_snapshot" :: held 0.969s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:22:05.750288 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Snapshots deleted, issuing volume delete {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/manager.py:1023}} Sep 27 10:22:05.758835 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:05.861339 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:22:05.862123 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Volume volume-e8e28128-e0c5-4f4a-80f2-ee68453209ee is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:22:05.871538 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] deleting rbd volume volume-e8e28128-e0c5-4f4a-80f2-ee68453209ee {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:22:06.403120 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Created reservations ['524f0af4-f23e-4fda-87d0-25b3aa190eeb', '24a1ce9a-795a-4f2b-9678-5eb5dbe0d8a6', 'c984ce9d-6719-4f3e-801f-d355d02b9bca', '28b0ed90-1ae0-41da-8fd8-6d0488e3182d'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:22:06.493201 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:06.557708 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:06.637262 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:22:06.659691 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Deleted volume successfully. Sep 27 10:22:06.695205 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d28959fe-ea52-49ba-bda2-ab5857565e36 tempest-VolumesDeleteCascade-1826627454 None] Lock "cinder-e8e28128-e0c5-4f4a-80f2-ee68453209ee-delete_volume" released by "delete_volume" :: held 3.079s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:22:11.335774 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:22:11.367460 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Flow 'volume_create_manager' (dfa626e9-0c77-44d6-b894-dc8988fdc448) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:22:11.377933 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (965d00c5-e24e-4d45-b518-9beebbd9464c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:11.405668 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (965d00c5-e24e-4d45-b518-9beebbd9464c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:22:11Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneNegativeTest-Volume-2045862849',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=911b61c5-981b-4445-8861-1b780d54c362,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='87e1be172b4845fe9ad5333d323fff8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:22:11Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:22:11Z,use_quota=True,user_id='f2cc166dabae4adc8f28b755de18f684',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:11.428893 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cb5f4854-b3ba-45bb-b400-5feaef5522a2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:11.433715 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cb5f4854-b3ba-45bb-b400-5feaef5522a2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:11.440127 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (754d4d8d-71d5-4806-bdc7-7ca3dfd8d9bd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:11.445330 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (754d4d8d-71d5-4806-bdc7-7ca3dfd8d9bd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '911b61c5-981b-4445-8861-1b780d54c362', 'volume_name': 'volume-911b61c5-981b-4445-8861-1b780d54c362', 'volume_size': 2}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:11.452122 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e9ff86da-40a3-4fe4-84c5-ca3f7eab2d27) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:11.457260 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e9ff86da-40a3-4fe4-84c5-ca3f7eab2d27) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:11.463036 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (781953f5-21e8-4cea-bc14-f340ea05e76e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:11.464347 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Volume 911b61c5-981b-4445-8861-1b780d54c362: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-911b61c5-981b-4445-8861-1b780d54c362', 'volume_size': 2} Sep 27 10:22:11.465260 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] creating volume 'volume-911b61c5-981b-4445-8861-1b780d54c362' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:22:11.466859 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:11.548540 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:22:11.549336 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:22:11.565163 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (781953f5-21e8-4cea-bc14-f340ea05e76e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-911b61c5-981b-4445-8861-1b780d54c362', 'volume_size': 2}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:11.573664 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1ffd5f6b-7106-4e85-9d69-a899c89af4ed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:11.585938 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Volume volume-911b61c5-981b-4445-8861-1b780d54c362 (911b61c5-981b-4445-8861-1b780d54c362): created successfully Sep 27 10:22:11.591056 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1ffd5f6b-7106-4e85-9d69-a899c89af4ed) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:11.623530 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Flow 'volume_create_manager' (dfa626e9-0c77-44d6-b894-dc8988fdc448) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:22:11.659307 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-fed638b9-c8e2-482f-adc9-79929dceb300 tempest-VolumesCloneNegativeTest-2095545067 None] Created volume successfully. Sep 27 10:22:12.591057 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] Acquiring lock "cinder-911b61c5-981b-4445-8861-1b780d54c362-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:22:12.620963 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] Lock "cinder-911b61c5-981b-4445-8861-1b780d54c362-delete_volume" acquired by "delete_volume" :: waited 0.030s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:22:12.674330 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:12.727952 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:22:12.728374 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] Volume volume-911b61c5-981b-4445-8861-1b780d54c362 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:22:12.740425 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] deleting rbd volume volume-911b61c5-981b-4445-8861-1b780d54c362 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:22:13.204714 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] Created reservations ['505cd4db-1b2a-42d0-a25e-2419fe1d9b6e', '02c493e7-2ade-4cda-abe3-da8fbc91bdc5', '57cb49da-622e-46fe-a1a9-dd0a76971bdf', '9fa78e75-b1f8-4f15-8bb6-ae84ff3dff7d'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:22:13.238080 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=66,cinder:UPDATE=27,cinder:DELETE=1,cinder:INSERT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:13.256798 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=56,cinder:DELETE=4,cinder:UPDATE=30,cinder:INSERT=10 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:13.320292 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:13.402786 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:13.495895 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:22:13.501752 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] Deleted volume successfully. Sep 27 10:22:13.535828 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-cd864fd9-edd4-404b-8c5b-8ece7ca34a08 tempest-VolumesCloneNegativeTest-2095545067 None] Lock "cinder-911b61c5-981b-4445-8861-1b780d54c362-delete_volume" released by "delete_volume" :: held 0.913s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:22:17.994606 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:22:18.078418 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Flow 'volume_create_manager' (0588544e-0ec7-4159-8b8c-d88e864f1284) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:22:18.107165 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (29543d4f-6cfc-431a-9f9b-05efc571749d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:18.146229 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (29543d4f-6cfc-431a-9f9b-05efc571749d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:22:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesNegativeTest-Volume-1996023182',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6dd0b368-0b4b-4d18-8dbc-b8e153e13410,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ab5aec6b80a447bab8eb4f1adf06e61f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:22:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:22:18Z,use_quota=True,user_id='b652b17bea8942c4b4a30034c6c1f44c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:18.160774 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ceaaa44f-eb48-40f3-9eb0-c71bc2cfb971) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:18.186947 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ceaaa44f-eb48-40f3-9eb0-c71bc2cfb971) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:18.200092 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (aaeb4ca2-5e9c-4697-8705-d19feff09a77) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:18.214390 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (aaeb4ca2-5e9c-4697-8705-d19feff09a77) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6dd0b368-0b4b-4d18-8dbc-b8e153e13410', 'volume_name': 'volume-6dd0b368-0b4b-4d18-8dbc-b8e153e13410', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:18.232675 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a79c6753-9652-46bc-add7-78e46ec6c97a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:18.248481 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a79c6753-9652-46bc-add7-78e46ec6c97a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:18.276581 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e24e2b3d-22a6-4611-b7a2-4a9225ef5d55) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:18.284985 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Volume 6dd0b368-0b4b-4d18-8dbc-b8e153e13410: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6dd0b368-0b4b-4d18-8dbc-b8e153e13410', 'volume_size': 1} Sep 27 10:22:18.285738 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] creating volume 'volume-6dd0b368-0b4b-4d18-8dbc-b8e153e13410' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:22:18.291743 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:18.483063 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:22:18.484360 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:22:18.492333 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e24e2b3d-22a6-4611-b7a2-4a9225ef5d55) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6dd0b368-0b4b-4d18-8dbc-b8e153e13410', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:18.498039 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (07bcba2a-7836-45c8-a122-59755952c00d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:22:18.515981 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Volume volume-6dd0b368-0b4b-4d18-8dbc-b8e153e13410 (6dd0b368-0b4b-4d18-8dbc-b8e153e13410): created successfully Sep 27 10:22:18.520947 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (07bcba2a-7836-45c8-a122-59755952c00d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:22:18.528939 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Flow 'volume_create_manager' (0588544e-0ec7-4159-8b8c-d88e864f1284) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:22:18.553218 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-df5efe9a-e968-4f14-8fd4-b46e3b9beda0 tempest-VolumesNegativeTest-72830281 None] Created volume successfully. Sep 27 10:22:29.770768 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:22:29.771902 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:29.850185 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:29.926520 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:22:46.335860 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=14,cinder:UPDATE=9,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:46.338057 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=9,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:46.385282 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 req-e726b7b1-09cf-47db-984b-1916ba7e22cc tempest-AttachVolumeShelveTestJSON-1912240963 None] Terminate volume connection completed successfully. Sep 27 10:22:46.385772 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 req-e726b7b1-09cf-47db-984b-1916ba7e22cc tempest-AttachVolumeShelveTestJSON-1912240963 None] Deleting attachment d41b920c-93e8-433a-8ece-d27e2d50c153. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:22:58.357351 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] Acquiring lock "cinder-6dd0b368-0b4b-4d18-8dbc-b8e153e13410-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:22:58.405763 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] Lock "cinder-6dd0b368-0b4b-4d18-8dbc-b8e153e13410-delete_volume" acquired by "delete_volume" :: waited 0.048s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:22:58.540734 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:58.663015 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:22:58.663435 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] Volume volume-6dd0b368-0b4b-4d18-8dbc-b8e153e13410 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:22:58.680603 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] deleting rbd volume volume-6dd0b368-0b4b-4d18-8dbc-b8e153e13410 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:22:59.206488 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] Created reservations ['81bb195b-f0e2-47d7-ad52-668f8ebe7c86', '95aa8669-90bb-41a6-b0d5-0dbd91d0b773', 'ff694c97-d99b-432e-9aba-68200ef7a0f6', '13b27f9f-7af2-44ab-acd5-ef697260aede'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:22:59.304191 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:59.382849 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:22:59.452518 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:22:59.458689 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] Deleted volume successfully. Sep 27 10:22:59.484033 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5d0e3836-61c1-47e8-878f-e6431f329d8c tempest-VolumesNegativeTest-72830281 None] Lock "cinder-6dd0b368-0b4b-4d18-8dbc-b8e153e13410-delete_volume" released by "delete_volume" :: held 1.078s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:23:17.042798 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=18,cinder:UPDATE=15,cinder:INSERT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:17.043871 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=19,cinder:UPDATE=4,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:29.776227 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:23:29.777526 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:23:29.869503 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:23:29.952320 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:23:37.053125 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:37.054173 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:47.079704 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:55.891212 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:23:55.924112 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Flow 'volume_create_manager' (9331a078-e6ff-4d4b-bd6e-8369ef00f84a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:23:55.934713 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (206f6541-8a73-414c-82ba-fb9421975bdf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:23:55.956484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (206f6541-8a73-414c-82ba-fb9421975bdf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:23:55Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AbsoluteLimitsTests-Volume-1948553133',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=7e5b107b-baa1-4897-bb5a-261d4fd73047,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8f280feb1d96433dbbfb114f3d5bf8d1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:23:56Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:23:56Z,use_quota=True,user_id='b1a233f92d6743c0a7e55730ac0af920',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:23:55.964557 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1fce1f5c-0166-420f-9571-3e31d6a7642a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:23:55.970508 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1fce1f5c-0166-420f-9571-3e31d6a7642a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:23:55.977398 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (82fac18f-79ff-4e8d-b752-36a50d335afe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:23:55.982221 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (82fac18f-79ff-4e8d-b752-36a50d335afe) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '7e5b107b-baa1-4897-bb5a-261d4fd73047', 'volume_name': 'volume-7e5b107b-baa1-4897-bb5a-261d4fd73047', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:23:55.989989 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0d0dbdc5-6ccc-4218-82f5-f2a08bff12f3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:23:55.995521 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0d0dbdc5-6ccc-4218-82f5-f2a08bff12f3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:23:56.001547 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d734fca8-7453-4395-aab7-4d3ce9360221) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:23:56.003521 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Volume 7e5b107b-baa1-4897-bb5a-261d4fd73047: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-7e5b107b-baa1-4897-bb5a-261d4fd73047', 'volume_size': 1} Sep 27 10:23:56.003521 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] creating volume 'volume-7e5b107b-baa1-4897-bb5a-261d4fd73047' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:23:56.005373 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:23:56.113867 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:23:56.114548 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:23:56.121735 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d734fca8-7453-4395-aab7-4d3ce9360221) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7e5b107b-baa1-4897-bb5a-261d4fd73047', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:23:56.134299 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fa5351f3-fb46-47b1-9c12-b90a4567e853) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:23:56.156006 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Volume volume-7e5b107b-baa1-4897-bb5a-261d4fd73047 (7e5b107b-baa1-4897-bb5a-261d4fd73047): created successfully Sep 27 10:23:56.172110 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fa5351f3-fb46-47b1-9c12-b90a4567e853) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:23:56.196801 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Flow 'volume_create_manager' (9331a078-e6ff-4d4b-bd6e-8369ef00f84a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:23:56.214043 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1b7604ae-60ff-4549-92ae-830daa57a2ef tempest-AbsoluteLimitsTests-768105408 None] Created volume successfully. Sep 27 10:23:57.111588 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] Acquiring lock "cinder-7e5b107b-baa1-4897-bb5a-261d4fd73047-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:23:57.153103 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] Lock "cinder-7e5b107b-baa1-4897-bb5a-261d4fd73047-delete_volume" acquired by "delete_volume" :: waited 0.041s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:23:57.256483 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:23:57.346553 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:23:57.347407 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] Volume volume-7e5b107b-baa1-4897-bb5a-261d4fd73047 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:23:57.362556 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] deleting rbd volume volume-7e5b107b-baa1-4897-bb5a-261d4fd73047 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:23:57.940341 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] Created reservations ['766a102d-dc4e-4f7b-a27e-d2bd0eb5493f', 'eb29a362-cb4a-475b-8bdc-c1915fad7964', '4d95948b-b265-4c74-bf05-76106396e4b4', '6dd7e2f0-91d3-4faf-bd79-59aef180f0a0'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:23:58.042737 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:23:58.110852 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:23:58.206471 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:23:58.212215 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] Deleted volume successfully. Sep 27 10:23:58.237831 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-15cfb6b4-f69e-48b4-8835-c9727dc12044 tempest-AbsoluteLimitsTests-768105408 None] Lock "cinder-7e5b107b-baa1-4897-bb5a-261d4fd73047-delete_volume" released by "delete_volume" :: held 1.085s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:24:13.685154 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=30,cinder:UPDATE=12,cinder:INSERT=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:13.693973 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:24:13.716732 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (48dbb487-5b4f-433b-8250-4abac07fe9b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:13.726251 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (50868425-043b-4506-9efe-5d360628dc0a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:13.752487 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (50868425-043b-4506-9efe-5d360628dc0a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:24:13Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-334931515',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cd03ad33-8f4c-4084-8b36-72d1fe84c526,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='712c1c4c858b4b788a7703345716504b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:24:14Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:24:14Z,use_quota=True,user_id='584c395757a54d9b9a64dd399d8557b1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:13.767034 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3ec465b9-c637-4e44-b69a-2f524323c77e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:13.771716 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3ec465b9-c637-4e44-b69a-2f524323c77e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:13.778552 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (25773e35-94b4-4bcb-bf30-ccfda709ac8a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:13.786698 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (25773e35-94b4-4bcb-bf30-ccfda709ac8a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'cd03ad33-8f4c-4084-8b36-72d1fe84c526', 'volume_name': 'volume-cd03ad33-8f4c-4084-8b36-72d1fe84c526', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:13.791610 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (324c7cb7-9b9e-4b0c-9ea3-9f919d3f6112) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:13.796269 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (324c7cb7-9b9e-4b0c-9ea3-9f919d3f6112) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:13.806023 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (76bca3df-8516-45b8-9a38-11a68078434c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:13.807525 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Volume cd03ad33-8f4c-4084-8b36-72d1fe84c526: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-cd03ad33-8f4c-4084-8b36-72d1fe84c526', 'volume_size': 1} Sep 27 10:24:13.809335 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] creating volume 'volume-cd03ad33-8f4c-4084-8b36-72d1fe84c526' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:24:13.811528 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:13.913537 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:24:13.914145 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:24:13.921033 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (76bca3df-8516-45b8-9a38-11a68078434c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cd03ad33-8f4c-4084-8b36-72d1fe84c526', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:13.927776 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (30eb666a-be15-4012-8244-a7e00d42abe6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:13.942537 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Volume volume-cd03ad33-8f4c-4084-8b36-72d1fe84c526 (cd03ad33-8f4c-4084-8b36-72d1fe84c526): created successfully Sep 27 10:24:13.949136 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (30eb666a-be15-4012-8244-a7e00d42abe6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:13.957553 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (48dbb487-5b4f-433b-8250-4abac07fe9b9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:13.972540 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e7c41108-baa5-4c77-a133-8b3e9caed808 tempest-VolumesSnapshotTestJSON-1705236581 None] Created volume successfully. Sep 27 10:24:15.054219 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a886a164-9868-4a4c-9057-cb71ddc5486d tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:16.122734 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:24:16.172576 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (1c3ba99e-807d-41d7-996c-5fc1ed7fa456) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:16.197706 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (dd6bccf6-fc22-49a4-8bee-66237a160cbe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:16.225502 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (dd6bccf6-fc22-49a4-8bee-66237a160cbe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:24:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1164677008',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4883ce9b-1665-4ab3-a1c3-b3bd68e272a9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='64f8fe93425d4e97b1ae5661172f4846',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:24:16Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:24:16Z,use_quota=True,user_id='178956f6c8154a06a0710b295aff3086',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:16.231434 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (22324207-ea0e-4fc1-abf2-194fa2177390) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:16.235831 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (22324207-ea0e-4fc1-abf2-194fa2177390) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:16.241596 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3f3f9ca4-eedf-4772-a951-87e45410b3ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:16.369718 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3f3f9ca4-eedf-4772-a951-87e45410b3ff) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '4883ce9b-1665-4ab3-a1c3-b3bd68e272a9', 'volume_name': 'volume-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:16.376131 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (db9ba553-fb59-4c0f-a996-3174a5899c4c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:16.380970 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (db9ba553-fb59-4c0f-a996-3174a5899c4c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:16.387224 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (11935bb0-286d-41b1-811b-bfa24e8e7270) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:16.388719 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Volume 4883ce9b-1665-4ab3-a1c3-b3bd68e272a9: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:24:16.389492 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Cloning 4883ce9b-1665-4ab3-a1c3-b3bd68e272a9 from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:24:16.390955 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:24:16.408524 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.018s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:24:16.416488 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-a886a164-9868-4a4c-9057-cb71ddc5486d tempest-VolumesSnapshotTestJSON-1705236581 None] Create snapshot completed successfully Sep 27 10:24:16.424482 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 24, 16, 413141)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:24:16.425258 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:24:16.425903 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:24:16.438734 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.030s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:24:16.440002 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:24:16.449162 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 24, 16, 446058)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:24:16.449878 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:24:16.450581 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:24:16.465357 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:16.594255 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] creating snapshot='volume-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:24:17.262929 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9.clone_snap' to 'volume-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:24:17.271587 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=28,cinder:UPDATE=14,cinder:DELETE=3,cinder:INSERT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:17.487666 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:24:17.488969 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:24:17.489430 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:24:17.533515 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:24:17.534164 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Marking volume 4883ce9b-1665-4ab3-a1c3-b3bd68e272a9 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:24:17.557837 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 4883ce9b-1665-4ab3-a1c3-b3bd68e272a9. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:24:17.558304 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Creating volume glance metadata for volume 4883ce9b-1665-4ab3-a1c3-b3bd68e272a9 backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:24:17.692160 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.driver [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] Creating a new backup for volume volume-cd03ad33-8f4c-4084-8b36-72d1fe84c526. {{(pid=138473) _get_backup_volume_temp_volume /opt/stack/cinder/cinder/volume/driver.py:1218}} Sep 27 10:24:17.777052 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] cloning volumes/volume-cd03ad33-8f4c-4084-8b36-72d1fe84c526@snapshot-65d29fc0-a3b9-4942-8752-0c17e1a6c717 to volume-b11c64da-b3d6-45df-8165-5b0be9a6ea7e {{(pid=138473) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 27 10:24:17.778871 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:17.869159 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:17.933986 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (11935bb0-286d-41b1-811b-bfa24e8e7270) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:17.940263 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (593d81fb-a6d4-4395-80a1-a8e391c4ecc4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:17.977691 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Volume volume-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9 (4883ce9b-1665-4ab3-a1c3-b3bd68e272a9): created successfully Sep 27 10:24:17.986479 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (593d81fb-a6d4-4395-80a1-a8e391c4ecc4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:18.000591 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (1c3ba99e-807d-41d7-996c-5fc1ed7fa456) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:18.015338 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-8b3fb7ec-62ae-469f-80af-b6629fae2d9d tempest-TestVolumeBootPattern-4600220 None] Created volume successfully. Sep 27 10:24:18.092132 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:24:18.092756 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:24:18.129797 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-27T10:24:17Z,data_timestamp=2022-09-27T10:24:15Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Backup-194702407',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=466b2609-b5c1-48f6-99ed-d589d60a69ce,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='712c1c4c858b4b788a7703345716504b',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id='65d29fc0-a3b9-4942-8752-0c17e1a6c717',status='creating',temp_snapshot_id=None,temp_volume_id='b11c64da-b3d6-45df-8165-5b0be9a6ea7e',updated_at=None,user_id='584c395757a54d9b9a64dd399d8557b1',volume_id=cd03ad33-8f4c-4084-8b36-72d1fe84c526) Sep 27 10:24:18.136231 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.backup.rpcapi [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] continue_backup in rpcapi backup_id 466b2609-b5c1-48f6-99ed-d589d60a69ce {{(pid=138473) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 27 10:24:18.519039 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:24:18.917100 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6924d1db-9443-420c-bc52-6a9ffc2804d0 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:19.730072 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.211s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:24:19.730700 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-b11c64da-b3d6-45df-8165-5b0be9a6ea7e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'b11c64da-b3d6-45df-8165-5b0be9a6ea7e', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:24:19.771216 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] Initialize volume connection completed successfully. Sep 27 10:24:19.994696 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6924d1db-9443-420c-bc52-6a9ffc2804d0 tempest-TestVolumeBootPattern-4600220 None] Create snapshot completed successfully Sep 27 10:24:26.057263 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c6477aa1-a100-4628-a2c2-67889cb6d1ff tempest-GroupsV320Test-838113681 None] Group tempest-GroupsV320Test-Group-895372343: creating Sep 27 10:24:26.159019 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c6477aa1-a100-4628-a2c2-67889cb6d1ff tempest-GroupsV320Test-838113681 None] Group tempest-GroupsV320Test-Group-895372343: created successfully Sep 27 10:24:26.203648 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:24:26.237974 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c6477aa1-a100-4628-a2c2-67889cb6d1ff tempest-GroupsV320Test-838113681 None] Create group completed successfully. Sep 27 10:24:26.289873 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (4abe3434-10b0-4468-a6ec-2ab854d33b82) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:26.300026 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e56e273b-61b7-47a7-89b3-5bcda767de99) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:26.340615 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e56e273b-61b7-47a7-89b3-5bcda767de99) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:24:26Z,deleted=False,deleted_at=None,display_description='',display_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=094fe8b7-e6a8-4b2c-adbc-dd8b86ced361,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='64f8fe93425d4e97b1ae5661172f4846',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:24:26Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=ad7a2197-c1e6-4e39-9521-89f6d7499b41,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:24:26Z,use_quota=True,user_id='178956f6c8154a06a0710b295aff3086',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:26.350765 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d37d6391-ede5-41c3-aeb9-3cb199c86861) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:26.375364 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d37d6391-ede5-41c3-aeb9-3cb199c86861) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:26.402129 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (533a51b5-2950-4f9f-8d37-0f77616004e9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:26.406206 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (533a51b5-2950-4f9f-8d37-0f77616004e9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '094fe8b7-e6a8-4b2c-adbc-dd8b86ced361', 'volume_name': 'volume-094fe8b7-e6a8-4b2c-adbc-dd8b86ced361', 'volume_size': 1, 'snapshot_id': 'ad7a2197-c1e6-4e39-9521-89f6d7499b41'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:26.435393 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cb746426-e50b-431c-ac24-be53b804539c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:26.439357 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cb746426-e50b-431c-ac24-be53b804539c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:26.452194 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ea06e265-0f86-4348-943f-41f4995fe73e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:26.453334 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Volume 094fe8b7-e6a8-4b2c-adbc-dd8b86ced361: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-094fe8b7-e6a8-4b2c-adbc-dd8b86ced361', 'volume_size': 1, 'snapshot_id': 'ad7a2197-c1e6-4e39-9521-89f6d7499b41'} Sep 27 10:24:26.499482 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] cloning volumes/volume-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9@snapshot-ad7a2197-c1e6-4e39-9521-89f6d7499b41 to volume-094fe8b7-e6a8-4b2c-adbc-dd8b86ced361 {{(pid=138473) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 27 10:24:26.500055 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:26.747740 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:27.729687 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:24:27.729687 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:24:27.763553 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Marking volume 094fe8b7-e6a8-4b2c-adbc-dd8b86ced361 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:24:27.790684 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Copying metadata from snapshot ad7a2197-c1e6-4e39-9521-89f6d7499b41 to 094fe8b7-e6a8-4b2c-adbc-dd8b86ced361. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:431}} Sep 27 10:24:27.944610 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ea06e265-0f86-4348-943f-41f4995fe73e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-094fe8b7-e6a8-4b2c-adbc-dd8b86ced361', 'volume_size': 1, 'snapshot_id': 'ad7a2197-c1e6-4e39-9521-89f6d7499b41'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:27.950317 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6cbe39be-b7e2-4538-9ee5-383b36952460) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:27.956242 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-4b70645f-1910-4333-884c-c81dda85e09d tempest-GroupsV320Test-838113681 None] Created reservations ['1750d738-7766-4e77-ba38-237b2376951b'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:24:27.993431 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Volume volume-094fe8b7-e6a8-4b2c-adbc-dd8b86ced361 (094fe8b7-e6a8-4b2c-adbc-dd8b86ced361): created successfully Sep 27 10:24:28.013010 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6cbe39be-b7e2-4538-9ee5-383b36952460) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:28.025705 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (4abe3434-10b0-4468-a6ec-2ab854d33b82) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:28.086538 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4b70645f-1910-4333-884c-c81dda85e09d tempest-GroupsV320Test-838113681 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:28.188741 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4b70645f-1910-4333-884c-c81dda85e09d tempest-GroupsV320Test-838113681 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:28.265875 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-4b70645f-1910-4333-884c-c81dda85e09d tempest-GroupsV320Test-838113681 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:24:28.278747 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-4b70645f-1910-4333-884c-c81dda85e09d tempest-GroupsV320Test-838113681 None] Delete group completed successfully. Sep 27 10:24:28.287478 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-2642dc0b-67ad-4bb8-bcda-f5ad5b6225cd tempest-TestVolumeBootPattern-4600220 None] Created volume successfully. Sep 27 10:24:29.774468 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:24:29.775900 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:29.883410 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:29.983632 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:24:29.999779 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-44c0c19b-0396-4138-8fe5-0cf27be29f92 tempest-TestVolumeBootPattern-4600220 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:24:31.204484 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-44c0c19b-0396-4138-8fe5-0cf27be29f92 tempest-TestVolumeBootPattern-4600220 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.209s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:24:31.205546 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-44c0c19b-0396-4138-8fe5-0cf27be29f92 tempest-TestVolumeBootPattern-4600220 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-094fe8b7-e6a8-4b2c-adbc-dd8b86ced361', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '094fe8b7-e6a8-4b2c-adbc-dd8b86ced361', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:24:31.221442 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=68,cinder:UPDATE=12,cinder:DELETE=2,cinder:INSERT=18 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:31.303968 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-44c0c19b-0396-4138-8fe5-0cf27be29f92 tempest-TestVolumeBootPattern-4600220 None] Connection info returned from driver {'name': 'volumes/volume-094fe8b7-e6a8-4b2c-adbc-dd8b86ced361', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '094fe8b7-e6a8-4b2c-adbc-dd8b86ced361', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'e666341b-6596-4fe7-8ed2-af5c26c5f20b'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:24:31.437286 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be req-44c0c19b-0396-4138-8fe5-0cf27be29f92 tempest-TestVolumeBootPattern-4600220 None] attachment_update completed successfully. Sep 27 10:24:35.427730 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=63,cinder:INSERT=21,cinder:UPDATE=14,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:35.457080 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] Terminate volume connection completed successfully. Sep 27 10:24:35.495907 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] Remove volume export completed successfully. Sep 27 10:24:35.533918 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-b11c64da-b3d6-45df-8165-5b0be9a6ea7e-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:24:35.558158 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-b11c64da-b3d6-45df-8165-5b0be9a6ea7e-delete_volume" acquired by "delete_volume" :: waited 0.024s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:24:35.587075 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:35.692964 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:24:35.704157 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] deleting rbd volume volume-b11c64da-b3d6-45df-8165-5b0be9a6ea7e {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:24:36.076814 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] Deleted volume successfully. Sep 27 10:24:36.083749 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5766fa20-0cfd-4285-babb-e5d111042e12 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-b11c64da-b3d6-45df-8165-5b0be9a6ea7e-delete_volume" released by "delete_volume" :: held 0.525s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:24:37.288963 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:24:37.311910 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Flow 'volume_create_manager' (8b9def41-d519-463d-841b-1e5dd644b0c2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:37.319602 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8bdd6cd7-4460-4cca-984c-218be2f1bde4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:37.347037 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8bdd6cd7-4460-4cca-984c-218be2f1bde4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:24:37Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasAdminTestJSON-Volume-177328586',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=94651318-bd5e-4b04-a2d6-407a07832119,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='47cb37050e9840688ff10a7c98c3e428',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:24:37Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:24:37Z,use_quota=True,user_id='f558b8405a9a4364887860f54d4ce225',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:37.355927 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0761705a-0082-476b-9d4c-40394c85babc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:37.363732 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0761705a-0082-476b-9d4c-40394c85babc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:37.372108 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e74ec735-90ef-4599-ada3-448383132f93) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:37.380241 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e74ec735-90ef-4599-ada3-448383132f93) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '94651318-bd5e-4b04-a2d6-407a07832119', 'volume_name': 'volume-94651318-bd5e-4b04-a2d6-407a07832119', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:37.385397 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a2e733d-98d2-4464-8a65-d1c3bda8f0bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:37.390070 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a2e733d-98d2-4464-8a65-d1c3bda8f0bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:37.396586 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (19b8c269-77d5-4568-bdad-65d47b2a8eb1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:37.397976 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Volume 94651318-bd5e-4b04-a2d6-407a07832119: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-94651318-bd5e-4b04-a2d6-407a07832119', 'volume_size': 1} Sep 27 10:24:37.398536 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] creating volume 'volume-94651318-bd5e-4b04-a2d6-407a07832119' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:24:37.399352 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:37.540245 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:24:37.541330 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:24:37.549575 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (19b8c269-77d5-4568-bdad-65d47b2a8eb1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-94651318-bd5e-4b04-a2d6-407a07832119', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:37.562162 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dcb8c03d-a0c4-49e7-97f8-e8bb87a50165) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:37.581992 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Volume volume-94651318-bd5e-4b04-a2d6-407a07832119 (94651318-bd5e-4b04-a2d6-407a07832119): created successfully Sep 27 10:24:37.586913 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dcb8c03d-a0c4-49e7-97f8-e8bb87a50165) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:37.596930 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Flow 'volume_create_manager' (8b9def41-d519-463d-841b-1e5dd644b0c2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:37.619888 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-808a9e4f-1b54-4182-ad80-2fed2dc645ab tempest-VolumeQuotasAdminTestJSON-669479620 None] Created volume successfully. Sep 27 10:24:38.455673 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] Acquiring lock "cinder-94651318-bd5e-4b04-a2d6-407a07832119-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:24:38.475425 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] Lock "cinder-94651318-bd5e-4b04-a2d6-407a07832119-delete_volume" acquired by "delete_volume" :: waited 0.020s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:24:38.553993 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:38.642115 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:24:38.642665 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] Volume volume-94651318-bd5e-4b04-a2d6-407a07832119 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:24:38.654055 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] deleting rbd volume volume-94651318-bd5e-4b04-a2d6-407a07832119 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:24:38.994437 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] Created reservations ['5b001a8c-8aaa-47a9-ac52-26892380f266', 'c46ed275-6616-475e-8609-9a06a6241b32', '9d5345eb-54b7-4ccc-83b4-a4db4cb7eab4', '0bc2e9e7-bdae-4bb8-a794-4b7b3ab87fe6'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:24:39.074897 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:39.142493 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:39.214667 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:24:39.222069 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] Deleted volume successfully. Sep 27 10:24:39.252058 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-2b7f6ad2-99dc-4bd2-be99-4699111fc5dc tempest-VolumeQuotasAdminTestJSON-669479620 None] Lock "cinder-94651318-bd5e-4b04-a2d6-407a07832119-delete_volume" released by "delete_volume" :: held 0.776s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:24:40.272752 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:24:40.297306 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Flow 'volume_create_manager' (f2f1a931-1ae5-4709-86fd-d2687660a594) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:40.309313 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d54793b3-ceec-4ebc-9482-c5c395363c1e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:40.351798 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d54793b3-ceec-4ebc-9482-c5c395363c1e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:24:40Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasAdminTestJSON-Volume-321255234',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=300fc5ce-a923-459d-89e7-e2c6705801df,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='47cb37050e9840688ff10a7c98c3e428',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:24:40Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:24:40Z,use_quota=True,user_id='f558b8405a9a4364887860f54d4ce225',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:40.366324 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1c1b711b-0ec8-421a-96f2-418ea4fcec23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:40.378333 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1c1b711b-0ec8-421a-96f2-418ea4fcec23) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:40.389103 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c8feacd9-2281-4bd5-8590-8e456c1a42a1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:40.400304 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c8feacd9-2281-4bd5-8590-8e456c1a42a1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '300fc5ce-a923-459d-89e7-e2c6705801df', 'volume_name': 'volume-300fc5ce-a923-459d-89e7-e2c6705801df', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:40.405701 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (604a7d60-e570-4f9d-8305-cc784ad78157) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:40.410769 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (604a7d60-e570-4f9d-8305-cc784ad78157) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:40.417577 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3dd47f48-5b0f-43d6-a6ba-804525362797) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:40.418816 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Volume 300fc5ce-a923-459d-89e7-e2c6705801df: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-300fc5ce-a923-459d-89e7-e2c6705801df', 'volume_size': 1} Sep 27 10:24:40.419206 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] creating volume 'volume-300fc5ce-a923-459d-89e7-e2c6705801df' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:24:40.419985 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:40.532305 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:24:40.532976 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:24:40.537658 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3dd47f48-5b0f-43d6-a6ba-804525362797) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-300fc5ce-a923-459d-89e7-e2c6705801df', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:40.543303 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3939dbc3-c4b8-44be-bae8-4472f4a07813) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:40.557818 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Volume volume-300fc5ce-a923-459d-89e7-e2c6705801df (300fc5ce-a923-459d-89e7-e2c6705801df): created successfully Sep 27 10:24:40.562175 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3939dbc3-c4b8-44be-bae8-4472f4a07813) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:40.568636 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Flow 'volume_create_manager' (f2f1a931-1ae5-4709-86fd-d2687660a594) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:40.582469 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-71cc1a42-2bf9-4494-8919-7e00b92e8a6f tempest-VolumeQuotasAdminTestJSON-669479620 None] Created volume successfully. Sep 27 10:24:42.119701 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-2dcc541c-ce52-4f16-9a1e-b457331ce8f2 tempest-VolumeQuotasAdminTestJSON-1705587746 None] Transfer volume completed successfully. Sep 27 10:24:42.602841 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] Acquiring lock "cinder-300fc5ce-a923-459d-89e7-e2c6705801df-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:24:42.622533 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] Lock "cinder-300fc5ce-a923-459d-89e7-e2c6705801df-delete_volume" acquired by "delete_volume" :: waited 0.020s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:24:42.659169 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:42.714309 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:24:42.714733 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] Volume volume-300fc5ce-a923-459d-89e7-e2c6705801df is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:24:42.722050 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] deleting rbd volume volume-300fc5ce-a923-459d-89e7-e2c6705801df {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:24:42.983471 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] Created reservations ['9830293f-73fb-4f65-9955-301bb7affbb5', '83c76c5c-b678-4f2b-8304-573b9f1ada77', '573519b6-98d2-4beb-9022-2bf45674fb7d', 'ff6befed-0fee-4de0-b498-2e43f741f6a1'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:24:43.060738 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:43.096611 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:43.128907 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:24:43.133128 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] Deleted volume successfully. Sep 27 10:24:43.149607 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a1f1c5e0-22aa-4b5e-856c-12d2cbaadfd9 tempest-VolumeQuotasAdminTestJSON-1415092905 None] Lock "cinder-300fc5ce-a923-459d-89e7-e2c6705801df-delete_volume" released by "delete_volume" :: held 0.527s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:24:45.564940 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-5afcaa30-c865-46d1-9be9-ecb0e1d113fd tempest-TestVolumeBootPattern-4600220 None] Terminate volume connection completed successfully. Sep 27 10:24:45.568272 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-5afcaa30-c865-46d1-9be9-ecb0e1d113fd tempest-TestVolumeBootPattern-4600220 None] Deleting attachment e666341b-6596-4fe7-8ed2-af5c26c5f20b. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:24:45.817760 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-6fd2ce9a-ed8a-4df3-8ca1-1e76e1eff83e tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-094fe8b7-e6a8-4b2c-adbc-dd8b86ced361-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:24:45.840891 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-6fd2ce9a-ed8a-4df3-8ca1-1e76e1eff83e tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-094fe8b7-e6a8-4b2c-adbc-dd8b86ced361-delete_volume" acquired by "delete_volume" :: waited 0.023s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:24:45.916505 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-6fd2ce9a-ed8a-4df3-8ca1-1e76e1eff83e tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:46.077100 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-6fd2ce9a-ed8a-4df3-8ca1-1e76e1eff83e tempest-TestVolumeBootPattern-4600220 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:24:46.088964 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-6fd2ce9a-ed8a-4df3-8ca1-1e76e1eff83e tempest-TestVolumeBootPattern-4600220 None] deleting rbd volume volume-094fe8b7-e6a8-4b2c-adbc-dd8b86ced361 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:24:46.525665 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-6fd2ce9a-ed8a-4df3-8ca1-1e76e1eff83e tempest-TestVolumeBootPattern-4600220 None] Created reservations ['0e48ad3c-948b-4347-912a-b5c8eaa7af1d', '5ac28d05-e90a-45da-89e9-cb8006efe9c0', '98991342-f052-486b-bd76-63c5db3e187e', '462d0094-3fc2-49db-9f28-04e72689bb83'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:24:46.593339 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-6fd2ce9a-ed8a-4df3-8ca1-1e76e1eff83e tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:46.649364 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-6fd2ce9a-ed8a-4df3-8ca1-1e76e1eff83e tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:46.708155 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-6fd2ce9a-ed8a-4df3-8ca1-1e76e1eff83e tempest-TestVolumeBootPattern-4600220 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:24:46.721632 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-6fd2ce9a-ed8a-4df3-8ca1-1e76e1eff83e tempest-TestVolumeBootPattern-4600220 None] Deleted volume successfully. Sep 27 10:24:46.748173 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c req-6fd2ce9a-ed8a-4df3-8ca1-1e76e1eff83e tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-094fe8b7-e6a8-4b2c-adbc-dd8b86ced361-delete_volume" released by "delete_volume" :: held 0.907s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:24:47.663359 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-f4bc1d57-7da6-47d3-a0a5-565d653a798f tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-ad7a2197-c1e6-4e39-9521-89f6d7499b41-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:24:47.683547 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-f4bc1d57-7da6-47d3-a0a5-565d653a798f tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-ad7a2197-c1e6-4e39-9521-89f6d7499b41-delete_snapshot" acquired by "delete_snapshot" :: waited 0.019s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:24:47.685561 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f4bc1d57-7da6-47d3-a0a5-565d653a798f tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:48.927522 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=63,cinder:UPDATE=29,cinder:INSERT=6,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:48.951759 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=56,cinder:UPDATE=35,cinder:DELETE=3,cinder:INSERT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:48.980948 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-f4bc1d57-7da6-47d3-a0a5-565d653a798f tempest-TestVolumeBootPattern-4600220 None] Created reservations ['23f2781e-ccb9-4d26-be7a-cb5b38b583c2', '04738b51-e9a2-4b95-a16d-04e58d9e76de', 'c33fbb03-5a43-414e-ba42-73642b5899d2', 'af94aff7-c91e-4461-8fc9-e22a2f00bd0b'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:24:49.101236 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-f4bc1d57-7da6-47d3-a0a5-565d653a798f tempest-TestVolumeBootPattern-4600220 None] Delete snapshot completed successfully. Sep 27 10:24:49.109769 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-f4bc1d57-7da6-47d3-a0a5-565d653a798f tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-ad7a2197-c1e6-4e39-9521-89f6d7499b41-delete_snapshot" released by "delete_snapshot" :: held 1.427s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:24:49.627244 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:24:49.648382 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Flow 'volume_create_manager' (c0ddf6d6-993a-46c0-9d82-2abed6415a14) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:49.661691 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e3c4eeb1-4872-4016-9c9d-b941aa0b7fb1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:49.683125 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e3c4eeb1-4872-4016-9c9d-b941aa0b7fb1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:24:49Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-1927785665',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d558400b-8728-4133-8e2d-d76be2e106d8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b590d2de576e41aab42f79e4daef9c63',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:24:50Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:24:50Z,use_quota=True,user_id='74878b715fc94b23bb139cb30d8d20c0',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:49.700369 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (078c6086-49e9-4d99-ba12-71fcce66c39c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:49.705052 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (078c6086-49e9-4d99-ba12-71fcce66c39c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:49.712347 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b9b1d48d-9f53-4ed8-877d-31ef97e2d423) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:49.717222 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b9b1d48d-9f53-4ed8-877d-31ef97e2d423) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd558400b-8728-4133-8e2d-d76be2e106d8', 'volume_name': 'volume-d558400b-8728-4133-8e2d-d76be2e106d8', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:49.722121 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f013e855-b4f1-41d9-93ba-40f4559a1577) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:49.727059 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f013e855-b4f1-41d9-93ba-40f4559a1577) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:49.731871 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (84e72454-db27-4f51-9eac-a3eddeeecdbf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:49.733202 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Volume d558400b-8728-4133-8e2d-d76be2e106d8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d558400b-8728-4133-8e2d-d76be2e106d8', 'volume_size': 1} Sep 27 10:24:49.733950 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] creating volume 'volume-d558400b-8728-4133-8e2d-d76be2e106d8' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:24:49.735033 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:49.860983 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:24:49.861572 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:24:49.873947 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (84e72454-db27-4f51-9eac-a3eddeeecdbf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d558400b-8728-4133-8e2d-d76be2e106d8', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:49.883691 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2abb0bea-1a1f-40f1-84a6-051e782c10a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:49.899374 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Volume volume-d558400b-8728-4133-8e2d-d76be2e106d8 (d558400b-8728-4133-8e2d-d76be2e106d8): created successfully Sep 27 10:24:49.907774 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2abb0bea-1a1f-40f1-84a6-051e782c10a3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:49.917325 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Flow 'volume_create_manager' (c0ddf6d6-993a-46c0-9d82-2abed6415a14) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:49.932861 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:24:49.947484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:24:49.963574 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c9c66dc0-c765-4c3a-9b8b-d2998574cc70 tempest-AttachVolumeShelveTestJSON-1912240963 None] Created volume successfully. Sep 27 10:24:50.019036 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:50.146970 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:24:50.159869 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] deleting rbd volume volume-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:24:50.495906 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:24:50.533069 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] deleting parent snapshot volume-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:24:51.594286 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:24:51.635861 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Flow 'volume_create_manager' (c85a2a32-2029-47c8-9db2-bc0446f853fc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:51.643670 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9e2e870c-bafd-4686-91e3-e1bb45f8b9fc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:51.680222 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9e2e870c-bafd-4686-91e3-e1bb45f8b9fc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:24:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeTypesAccessTest-Volume-2028973335',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=80c067b7-d9fc-4262-97ad-f57095f87657,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f633659fc9eb47f6bdb3ce49287650b2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:24:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:24:51Z,use_quota=True,user_id='2a5957e42b6c4d9fb0187fec69c00d56',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(49100801-66c2-4cc1-8551-08f141019d81),volume_type_id=49100801-66c2-4cc1-8551-08f141019d81)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:51.686814 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (32951a6b-b981-44fa-b634-1cf6bdd5bb89) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:51.697436 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (32951a6b-b981-44fa-b634-1cf6bdd5bb89) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:51.703303 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3d386d5b-39ce-4005-94e0-8df3d78e757b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:51.707464 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3d386d5b-39ce-4005-94e0-8df3d78e757b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '80c067b7-d9fc-4262-97ad-f57095f87657', 'volume_name': 'volume-80c067b7-d9fc-4262-97ad-f57095f87657', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:51.713106 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9e893f69-a98c-438f-830d-1d181e6f2cbe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:51.717165 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9e893f69-a98c-438f-830d-1d181e6f2cbe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:51.721854 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c9911ddc-479e-44ae-8478-ca3d132f3e58) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:51.722946 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Volume 80c067b7-d9fc-4262-97ad-f57095f87657: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-80c067b7-d9fc-4262-97ad-f57095f87657', 'volume_size': 1} Sep 27 10:24:51.724153 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] creating volume 'volume-80c067b7-d9fc-4262-97ad-f57095f87657' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:24:51.725241 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:51.784085 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] Created reservations ['f51c4fc2-f039-4100-aa88-ac9b68e7d5a1', 'a581ec75-6507-4204-80cd-241ee59fcc0c', 'c35c7e50-f15b-4728-9c31-bcb98cd35cc9', '28828599-9606-457c-bbd5-010192e70960'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:24:51.839313 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:24:51.839949 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:24:51.844059 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c9911ddc-479e-44ae-8478-ca3d132f3e58) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-80c067b7-d9fc-4262-97ad-f57095f87657', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:51.849137 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (507a1715-f7d2-4895-9d9d-b228b3780ba4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:51.869711 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Volume volume-80c067b7-d9fc-4262-97ad-f57095f87657 (80c067b7-d9fc-4262-97ad-f57095f87657): created successfully Sep 27 10:24:51.874217 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (507a1715-f7d2-4895-9d9d-b228b3780ba4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:51.882173 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Flow 'volume_create_manager' (c85a2a32-2029-47c8-9db2-bc0446f853fc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:51.898270 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-a4c92eec-8332-49d4-b49d-3e572599b027 tempest-VolumeTypesAccessTest-1589848115 None] Created volume successfully. Sep 27 10:24:51.905636 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:51.971670 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:52.025984 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:24:52.035468 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] Deleted volume successfully. Sep 27 10:24:52.055003 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7a4fec2b-4c2e-4b97-a1ca-c89d5f61e9ec tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-4883ce9b-1665-4ab3-a1c3-b3bd68e272a9-delete_volume" released by "delete_volume" :: held 2.107s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:24:53.110799 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] Acquiring lock "cinder-80c067b7-d9fc-4262-97ad-f57095f87657-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:24:53.125423 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:24:53.146817 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (ad22fa85-362e-425c-ace3-080480ab01a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:53.154896 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9fb6ffe0-f0ce-4947-92be-6ff5e9df0fa6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:53.172878 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] Lock "cinder-80c067b7-d9fc-4262-97ad-f57095f87657-delete_volume" acquired by "delete_volume" :: waited 0.062s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:24:53.181106 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9fb6ffe0-f0ce-4947-92be-6ff5e9df0fa6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:24:53Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1809353611',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c2bfd580-e62a-4f33-a8ea-3e0437752d94,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='64f8fe93425d4e97b1ae5661172f4846',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:24:53Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:24:53Z,use_quota=True,user_id='178956f6c8154a06a0710b295aff3086',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:53.186337 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (434df851-dcf0-4c18-bc82-d46898768758) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:53.190214 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (434df851-dcf0-4c18-bc82-d46898768758) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:53.198120 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7bcef522-bbbe-409c-8dd5-77079577630b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:53.255906 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:53.315098 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:24:53.315702 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] Volume volume-80c067b7-d9fc-4262-97ad-f57095f87657 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:24:53.321234 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] deleting rbd volume volume-80c067b7-d9fc-4262-97ad-f57095f87657 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:24:53.355983 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7bcef522-bbbe-409c-8dd5-77079577630b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'c2bfd580-e62a-4f33-a8ea-3e0437752d94', 'volume_name': 'volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:53.361718 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dbb87ba5-b548-4526-9700-805425529d7f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:53.366644 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dbb87ba5-b548-4526-9700-805425529d7f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:53.371182 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8a49756a-60ac-449a-af1b-8e5dcaa58f81) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:53.372416 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Volume c2bfd580-e62a-4f33-a8ea-3e0437752d94: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:24:53.373026 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Cloning c2bfd580-e62a-4f33-a8ea-3e0437752d94 from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:24:53.373780 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:24:53.383387 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:24:53.395385 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 24, 53, 388840)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:24:53.396085 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:24:53.396631 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:24:53.402817 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.019s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:24:53.404297 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:24:53.411890 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 24, 53, 409090)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:24:53.412312 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:24:53.412734 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:24:53.432458 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:53.503217 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] creating snapshot='volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:24:54.535051 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94.clone_snap' to 'volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:24:54.585822 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:24:54.604909 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (717964a6-ae8d-4967-9c06-a472a060869f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:54.611783 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (510db97e-f16d-4b93-92b1-a3bcc7d32a36) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:54.633012 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (510db97e-f16d-4b93-92b1-a3bcc7d32a36) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:24:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-895952710',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1962f1c2-cd1b-44c0-ab12-4f391cd8acf5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='712c1c4c858b4b788a7703345716504b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:24:54Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:24:54Z,use_quota=True,user_id='584c395757a54d9b9a64dd399d8557b1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:54.638895 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (584b1031-a220-4ae3-889d-416eeebdaade) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:54.643753 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (584b1031-a220-4ae3-889d-416eeebdaade) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:54.650778 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d2eccf5b-61f6-4a51-9940-66a9167684f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:54.655230 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1075033680 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1075033680, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:24:54.655819 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1075033680 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1075033680, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:24:54.675876 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] Created reservations ['61b110cd-e5ec-47c8-94e0-e69ae6b72413', 'ab6693b2-49b5-47d7-b111-2b4907c11220', 'd16ea9c4-61d2-4592-b6a9-eda573e4bfad', 'cc412285-f3fa-482e-bc8b-9b02f38a426e'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:24:54.743055 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:24:54.745142 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:24:54.745577 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:24:54.766014 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:24:54.768812 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Marking volume c2bfd580-e62a-4f33-a8ea-3e0437752d94 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:24:54.795716 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to c2bfd580-e62a-4f33-a8ea-3e0437752d94. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:24:54.796352 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Creating volume glance metadata for volume c2bfd580-e62a-4f33-a8ea-3e0437752d94 backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:24:54.850256 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:54.906193 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:54.959459 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:24:54.977321 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] Deleted volume successfully. Sep 27 10:24:54.989644 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d2eccf5b-61f6-4a51-9940-66a9167684f9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '1962f1c2-cd1b-44c0-ab12-4f391cd8acf5', 'volume_name': 'volume-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:54.995529 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (16729d9f-d42a-44de-b5a5-edcb2d5a7fac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:54.999719 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (16729d9f-d42a-44de-b5a5-edcb2d5a7fac) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:55.005736 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7d9b03ab-0eca-4a65-a211-033226b18fe9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:55.006997 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Volume 1962f1c2-cd1b-44c0-ab12-4f391cd8acf5: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:24:55.007571 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Cloning 1962f1c2-cd1b-44c0-ab12-4f391cd8acf5 from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:24:55.008320 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:24:55.021625 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.013s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:24:55.024018 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:INSERT=12,cinder:SELECT=51,cinder:UPDATE=36,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:55.029990 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8a49756a-60ac-449a-af1b-8e5dcaa58f81) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:55.034634 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fe2c4e28-2869-4411-8190-c7bec99c8497) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:55.052694 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Volume volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94 (c2bfd580-e62a-4f33-a8ea-3e0437752d94): created successfully Sep 27 10:24:55.057495 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fe2c4e28-2869-4411-8190-c7bec99c8497) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:55.064529 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (ad22fa85-362e-425c-ace3-080480ab01a3) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:55.070692 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6c5ba8c9-25c0-4736-a34a-7a6709de1da7 tempest-VolumeTypesAccessTest-1589848115 None] Lock "cinder-80c067b7-d9fc-4262-97ad-f57095f87657-delete_volume" released by "delete_volume" :: held 1.898s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:24:55.074740 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 24, 55, 45948)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:24:55.075221 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:24:55.075687 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:24:55.080619 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.059s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:24:55.083690 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:24:55.088161 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-96d6eee0-c592-4311-8609-24ac41ee2e68 tempest-TestVolumeBootPattern-4600220 None] Created volume successfully. Sep 27 10:24:55.094429 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 24, 55, 91117)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:24:55.094843 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:24:55.095303 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:24:55.111577 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:24:55.217979 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] creating snapshot='volume-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:24:55.629419 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5.clone_snap' to 'volume-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:24:55.850619 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:24:55.852120 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:24:55.852644 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:24:55.895342 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:24:55.895967 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Marking volume 1962f1c2-cd1b-44c0-ab12-4f391cd8acf5 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:24:55.910402 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 1962f1c2-cd1b-44c0-ab12-4f391cd8acf5. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:24:55.910996 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Creating volume glance metadata for volume 1962f1c2-cd1b-44c0-ab12-4f391cd8acf5 backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:24:55.936300 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=61,cinder:UPDATE=22,cinder:DELETE=4,cinder:INSERT=13 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:56.074517 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7d9b03ab-0eca-4a65-a211-033226b18fe9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:56.090691 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d192a4ee-7ff3-40e1-8dc0-43633ae9e33d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:24:56.117709 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Volume volume-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5 (1962f1c2-cd1b-44c0-ab12-4f391cd8acf5): created successfully Sep 27 10:24:56.121408 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d192a4ee-7ff3-40e1-8dc0-43633ae9e33d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:24:56.132947 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (717964a6-ae8d-4967-9c06-a472a060869f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:24:56.148312 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-70f93abc-1e52-4d69-ae68-62a512c0820d tempest-VolumesSnapshotTestJSON-1705236581 None] Created volume successfully. Sep 27 10:24:57.613930 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-bb6d12db-088d-42d0-8688-6ee42f079aa2 req-9e58bdcc-78c9-4a5d-89bf-b8809d43844c tempest-VolumesSnapshotTestJSON-1705236581 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:24:58.854065 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-bb6d12db-088d-42d0-8688-6ee42f079aa2 req-9e58bdcc-78c9-4a5d-89bf-b8809d43844c tempest-VolumesSnapshotTestJSON-1705236581 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.255s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:24:58.856972 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-bb6d12db-088d-42d0-8688-6ee42f079aa2 req-9e58bdcc-78c9-4a5d-89bf-b8809d43844c tempest-VolumesSnapshotTestJSON-1705236581 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '1962f1c2-cd1b-44c0-ab12-4f391cd8acf5', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:24:58.915773 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-bb6d12db-088d-42d0-8688-6ee42f079aa2 req-9e58bdcc-78c9-4a5d-89bf-b8809d43844c tempest-VolumesSnapshotTestJSON-1705236581 None] Connection info returned from driver {'name': 'volumes/volume-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '1962f1c2-cd1b-44c0-ab12-4f391cd8acf5', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '6830ae69-41cc-4e47-a6b1-d53f71736393'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:24:59.038999 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-bb6d12db-088d-42d0-8688-6ee42f079aa2 req-9e58bdcc-78c9-4a5d-89bf-b8809d43844c tempest-VolumesSnapshotTestJSON-1705236581 None] attachment_update completed successfully. Sep 27 10:24:59.543991 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c18a1228-907a-40f3-bdc0-e9242c388ae2 req-47249b1d-cb8f-4624-af3b-5f1bf39cbc61 tempest-TestVolumeBootPattern-4600220 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:25:01.014402 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c18a1228-907a-40f3-bdc0-e9242c388ae2 req-47249b1d-cb8f-4624-af3b-5f1bf39cbc61 tempest-TestVolumeBootPattern-4600220 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.470s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:25:01.015196 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-c18a1228-907a-40f3-bdc0-e9242c388ae2 req-47249b1d-cb8f-4624-af3b-5f1bf39cbc61 tempest-TestVolumeBootPattern-4600220 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'c2bfd580-e62a-4f33-a8ea-3e0437752d94', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:25:01.070481 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-c18a1228-907a-40f3-bdc0-e9242c388ae2 req-47249b1d-cb8f-4624-af3b-5f1bf39cbc61 tempest-TestVolumeBootPattern-4600220 None] Connection info returned from driver {'name': 'volumes/volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'c2bfd580-e62a-4f33-a8ea-3e0437752d94', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '3ca665a8-fa80-4866-8bd9-f57a4868eb01'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:25:01.146746 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-c18a1228-907a-40f3-bdc0-e9242c388ae2 req-47249b1d-cb8f-4624-af3b-5f1bf39cbc61 tempest-TestVolumeBootPattern-4600220 None] attachment_update completed successfully. Sep 27 10:25:01.284104 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0fe135dd-58f0-4c4e-b92d-7593e1145ed5 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:02.342168 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0fe135dd-58f0-4c4e-b92d-7593e1145ed5 tempest-VolumesSnapshotTestJSON-1705236581 None] Create snapshot completed successfully Sep 27 10:25:03.022484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:25:03.071899 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Flow 'volume_create_manager' (bc2d6692-6ba2-4b08-b66c-c2a3af241318) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:25:03.080589 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (affb871b-5485-48e9-95da-f51cf1688879) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:03.102560 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (affb871b-5485-48e9-95da-f51cf1688879) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:25:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1648509330',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bb8280cc-ed98-4857-a90d-aef277d7bdaf,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='430555edbc87497db7d7f79f2b891c04',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:25:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:25:03Z,use_quota=True,user_id='fbcfed06aa6b4177a4287064884125b2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:03.112716 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ff87957c-82a3-4a85-a3ad-3fd579b88a65) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:03.117702 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ff87957c-82a3-4a85-a3ad-3fd579b88a65) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:03.124892 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7c7a8715-167c-4ad3-a186-39559125e222) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:03.129948 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7c7a8715-167c-4ad3-a186-39559125e222) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'bb8280cc-ed98-4857-a90d-aef277d7bdaf', 'volume_name': 'volume-bb8280cc-ed98-4857-a90d-aef277d7bdaf', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:03.135922 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f6e8799a-0b1e-44d8-aa69-276ce123e307) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:03.141100 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f6e8799a-0b1e-44d8-aa69-276ce123e307) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:03.154501 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (24e23ff7-0298-4bd1-9164-562dccec3beb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:03.156586 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Volume bb8280cc-ed98-4857-a90d-aef277d7bdaf: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-bb8280cc-ed98-4857-a90d-aef277d7bdaf', 'volume_size': 1} Sep 27 10:25:03.158205 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] creating volume 'volume-bb8280cc-ed98-4857-a90d-aef277d7bdaf' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:25:03.160545 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:03.325472 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:25:03.326126 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:25:03.332877 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (24e23ff7-0298-4bd1-9164-562dccec3beb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bb8280cc-ed98-4857-a90d-aef277d7bdaf', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:03.340291 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3d8beb29-cf18-44a8-ac4b-66c6856b1d9c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:03.356328 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Volume volume-bb8280cc-ed98-4857-a90d-aef277d7bdaf (bb8280cc-ed98-4857-a90d-aef277d7bdaf): created successfully Sep 27 10:25:03.364878 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3d8beb29-cf18-44a8-ac4b-66c6856b1d9c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:03.372220 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Flow 'volume_create_manager' (bc2d6692-6ba2-4b08-b66c-c2a3af241318) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:25:03.385098 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-8bba05ed-35d5-477f-9475-8829f9eb221a tempest-VolumesBackupsTest-353079186 None] Created volume successfully. Sep 27 10:25:03.785903 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a69f16e2-8767-4f53-94ab-fe7b26dd2ef3 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:04.417819 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-a69f16e2-8767-4f53-94ab-fe7b26dd2ef3 tempest-VolumesSnapshotTestJSON-1705236581 None] Create snapshot completed successfully Sep 27 10:25:05.007082 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-776009aa-564c-480a-8a0e-7ed426c03a90 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:06.266787 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=66,cinder:UPDATE=6,cinder:INSERT=27,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:06.304978 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-776009aa-564c-480a-8a0e-7ed426c03a90 tempest-VolumesSnapshotTestJSON-1705236581 None] Create snapshot completed successfully Sep 27 10:25:07.127517 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-319ad74b-b858-4304-9cea-1cd4f504d662 tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-2e28f67f-ca3b-41a3-9186-4d52cc24b962-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:25:07.151890 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-319ad74b-b858-4304-9cea-1cd4f504d662 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-2e28f67f-ca3b-41a3-9186-4d52cc24b962-delete_snapshot" acquired by "delete_snapshot" :: waited 0.024s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:25:07.153436 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-319ad74b-b858-4304-9cea-1cd4f504d662 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:16.318224 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=53,cinder:DELETE=4,cinder:UPDATE=13,cinder:INSERT=20 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:22.726751 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-3df12c14-9082-4eac-bab9-fd9236c872eb req-6086f23d-15aa-4982-a4e8-c4a5c2ce7de9 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:24.110582 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-3df12c14-9082-4eac-bab9-fd9236c872eb req-6086f23d-15aa-4982-a4e8-c4a5c2ce7de9 tempest-TestVolumeBootPattern-4600220 None] Create snapshot completed successfully Sep 27 10:25:25.726577 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 req-a2d42af9-d68a-497b-9bd1-8a8ec8b46232 tempest-VolumesBackupsTest-353079186 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:25:27.369042 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 req-a2d42af9-d68a-497b-9bd1-8a8ec8b46232 tempest-VolumesBackupsTest-353079186 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.643s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:25:27.369844 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 req-a2d42af9-d68a-497b-9bd1-8a8ec8b46232 tempest-VolumesBackupsTest-353079186 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-bb8280cc-ed98-4857-a90d-aef277d7bdaf', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'bb8280cc-ed98-4857-a90d-aef277d7bdaf', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:25:27.408994 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 req-a2d42af9-d68a-497b-9bd1-8a8ec8b46232 tempest-VolumesBackupsTest-353079186 None] Connection info returned from driver {'name': 'volumes/volume-bb8280cc-ed98-4857-a90d-aef277d7bdaf', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'bb8280cc-ed98-4857-a90d-aef277d7bdaf', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'f9e651fd-29e6-4fa2-8783-8236bff8a58c'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:25:27.494880 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 req-a2d42af9-d68a-497b-9bd1-8a8ec8b46232 tempest-VolumesBackupsTest-353079186 None] attachment_update completed successfully. Sep 27 10:25:29.772221 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:25:29.774007 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:29.854496 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:29.935599 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:25:30.373974 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:25:30.493152 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (0ad14564-39f1-4fb0-ace1-5a1c190f0341) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:25:30.513133 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7c0da0c9-a788-4009-b939-e18e9663241c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:30.573116 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7c0da0c9-a788-4009-b939-e18e9663241c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:25:30Z,deleted=False,deleted_at=None,display_description='',display_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c0f96bfe-dd85-464f-97d3-93728dd4ca02,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='64f8fe93425d4e97b1ae5661172f4846',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:25:30Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=b3c99c48-92e3-4590-aff4-92ad11f3158c,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:25:30Z,use_quota=True,user_id='178956f6c8154a06a0710b295aff3086',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:30.586503 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (108ad620-bcbd-4907-9473-b2cc62f6617c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:30.597860 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (108ad620-bcbd-4907-9473-b2cc62f6617c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:30.611982 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5b8554b5-d189-4455-b38d-c4a3d02ecb96) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:30.619818 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5b8554b5-d189-4455-b38d-c4a3d02ecb96) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'c0f96bfe-dd85-464f-97d3-93728dd4ca02', 'volume_name': 'volume-c0f96bfe-dd85-464f-97d3-93728dd4ca02', 'volume_size': 1, 'snapshot_id': 'b3c99c48-92e3-4590-aff4-92ad11f3158c'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:30.633105 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8c54f294-e985-41bb-b458-4cdca133f71c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:30.641065 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8c54f294-e985-41bb-b458-4cdca133f71c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:30.648466 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (39b15ea9-182c-48c4-8361-ec062110e62a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:30.650060 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Volume c0f96bfe-dd85-464f-97d3-93728dd4ca02: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-c0f96bfe-dd85-464f-97d3-93728dd4ca02', 'volume_size': 1, 'snapshot_id': 'b3c99c48-92e3-4590-aff4-92ad11f3158c'} Sep 27 10:25:30.691698 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7c622a50-4e5c-40b5-bd2e-3b95fe913bb9 tempest-VolumesBackupsTest-353079186 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-27T10:25:30Z,data_timestamp=2022-09-27T10:25:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-288395517',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=b99cbf98-71cd-42ee-9bd9-642c4070f1e8,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='430555edbc87497db7d7f79f2b891c04',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='fbcfed06aa6b4177a4287064884125b2',volume_id=bb8280cc-ed98-4857-a90d-aef277d7bdaf) Sep 27 10:25:30.694873 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.backup.rpcapi [None req-7c622a50-4e5c-40b5-bd2e-3b95fe913bb9 tempest-VolumesBackupsTest-353079186 None] continue_backup in rpcapi backup_id b99cbf98-71cd-42ee-9bd9-642c4070f1e8 {{(pid=138473) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 27 10:25:30.746644 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] cloning volumes/volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94@snapshot-b3c99c48-92e3-4590-aff4-92ad11f3158c to volume-c0f96bfe-dd85-464f-97d3-93728dd4ca02 {{(pid=138473) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 27 10:25:30.747615 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:30.888351 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:30.966530 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-7c622a50-4e5c-40b5-bd2e-3b95fe913bb9 tempest-VolumesBackupsTest-353079186 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:25:31.130711 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:25:31.131136 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:25:31.198566 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Marking volume c0f96bfe-dd85-464f-97d3-93728dd4ca02 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:25:31.221751 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Copying metadata from snapshot b3c99c48-92e3-4590-aff4-92ad11f3158c to c0f96bfe-dd85-464f-97d3-93728dd4ca02. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:431}} Sep 27 10:25:31.302885 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (39b15ea9-182c-48c4-8361-ec062110e62a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c0f96bfe-dd85-464f-97d3-93728dd4ca02', 'volume_size': 1, 'snapshot_id': 'b3c99c48-92e3-4590-aff4-92ad11f3158c'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:31.308052 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2c856b32-287c-4ef2-a3e1-631b62d63639) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:31.333921 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Volume volume-c0f96bfe-dd85-464f-97d3-93728dd4ca02 (c0f96bfe-dd85-464f-97d3-93728dd4ca02): created successfully Sep 27 10:25:31.338201 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2c856b32-287c-4ef2-a3e1-631b62d63639) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:31.344255 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (0ad14564-39f1-4fb0-ace1-5a1c190f0341) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:25:31.398063 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-eec4cd75-3d0d-442c-9e4e-38f3581062b4 tempest-TestVolumeBootPattern-4600220 None] Created volume successfully. Sep 27 10:25:32.201251 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-7c622a50-4e5c-40b5-bd2e-3b95fe913bb9 tempest-VolumesBackupsTest-353079186 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.235s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:25:32.202291 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7c622a50-4e5c-40b5-bd2e-3b95fe913bb9 tempest-VolumesBackupsTest-353079186 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-bb8280cc-ed98-4857-a90d-aef277d7bdaf', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'bb8280cc-ed98-4857-a90d-aef277d7bdaf', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:25:32.218430 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7c622a50-4e5c-40b5-bd2e-3b95fe913bb9 tempest-VolumesBackupsTest-353079186 None] Initialize volume connection completed successfully. Sep 27 10:25:33.738808 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-9876d234-ce54-46bd-90d0-89b5e343f60b tempest-TestVolumeBootPattern-4600220 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:25:34.796459 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7c622a50-4e5c-40b5-bd2e-3b95fe913bb9 tempest-VolumesBackupsTest-353079186 None] Terminate volume connection completed successfully. Sep 27 10:25:34.837641 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7c622a50-4e5c-40b5-bd2e-3b95fe913bb9 tempest-VolumesBackupsTest-353079186 None] Remove volume export completed successfully. Sep 27 10:25:34.841062 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-9876d234-ce54-46bd-90d0-89b5e343f60b tempest-TestVolumeBootPattern-4600220 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.103s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:25:34.842285 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-9876d234-ce54-46bd-90d0-89b5e343f60b tempest-TestVolumeBootPattern-4600220 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-c0f96bfe-dd85-464f-97d3-93728dd4ca02', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'c0f96bfe-dd85-464f-97d3-93728dd4ca02', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:25:34.879730 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-9876d234-ce54-46bd-90d0-89b5e343f60b tempest-TestVolumeBootPattern-4600220 None] Connection info returned from driver {'name': 'volumes/volume-c0f96bfe-dd85-464f-97d3-93728dd4ca02', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'c0f96bfe-dd85-464f-97d3-93728dd4ca02', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '36589d4b-1385-437d-833e-3af319d83d7c'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:25:34.933293 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 req-9876d234-ce54-46bd-90d0-89b5e343f60b tempest-TestVolumeBootPattern-4600220 None] attachment_update completed successfully. Sep 27 10:25:35.814125 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-4aa9143b-fcef-4b76-91bc-67178b911c6f req-298f76f5-5ff9-4d7e-b3e0-86ae7eebdc44 tempest-AttachVolumeShelveTestJSON-1912240963 None] No connector for attachment 53e97bd8-23aa-45b0-a297-3ab45d35d00d; skipping storage backend terminate_connection call. {{(pid=138473) _connection_terminate /opt/stack/cinder/cinder/volume/manager.py:4953}} Sep 27 10:25:35.814749 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-4aa9143b-fcef-4b76-91bc-67178b911c6f req-298f76f5-5ff9-4d7e-b3e0-86ae7eebdc44 tempest-AttachVolumeShelveTestJSON-1912240963 None] Deleting attachment 53e97bd8-23aa-45b0-a297-3ab45d35d00d. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:25:37.322912 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=58,cinder:UPDATE=6,cinder:INSERT=13 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:37.387070 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-319ad74b-b858-4304-9cea-1cd4f504d662 tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['62532167-fc30-49a0-a41b-df2dcb6cb96c', '3b6d22e4-6614-4a96-888c-d76126245dff', 'c71d0890-4cd7-4b7a-b7e5-d53ac91b1034', '383c565a-03d3-4fc4-a4f4-3fd1270df657'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:25:37.442419 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=70,cinder:UPDATE=16,cinder:INSERT=12,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:37.456447 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-319ad74b-b858-4304-9cea-1cd4f504d662 tempest-VolumesSnapshotTestJSON-1705236581 None] Delete snapshot completed successfully. Sep 27 10:25:37.465673 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-319ad74b-b858-4304-9cea-1cd4f504d662 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-2e28f67f-ca3b-41a3-9186-4d52cc24b962-delete_snapshot" released by "delete_snapshot" :: held 30.313s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:25:37.860238 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0754ed73-1d74-48d7-a293-bd48b8c64aa4 tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-ce92b4af-4c17-43ea-af12-e6ccaf22b49a-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:25:37.888869 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0754ed73-1d74-48d7-a293-bd48b8c64aa4 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-ce92b4af-4c17-43ea-af12-e6ccaf22b49a-delete_snapshot" acquired by "delete_snapshot" :: waited 0.032s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:25:37.891637 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0754ed73-1d74-48d7-a293-bd48b8c64aa4 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:38.994415 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-de12a93c-ad0a-45a9-b74b-5221780756ff req-362bb90c-b21b-4729-83cc-49fa52dc9611 tempest-VolumesBackupsTest-353079186 None] Terminate volume connection completed successfully. Sep 27 10:25:38.996140 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-de12a93c-ad0a-45a9-b74b-5221780756ff req-362bb90c-b21b-4729-83cc-49fa52dc9611 tempest-VolumesBackupsTest-353079186 None] Deleting attachment f9e651fd-29e6-4fa2-8783-8236bff8a58c. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:25:44.503786 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-44604b33-ee78-40cd-bb55-e8c90d9cfcdf tempest-VolumesBackupsTest-353079186 None] Acquiring lock "cinder-bb8280cc-ed98-4857-a90d-aef277d7bdaf-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:25:44.525366 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-44604b33-ee78-40cd-bb55-e8c90d9cfcdf tempest-VolumesBackupsTest-353079186 None] Lock "cinder-bb8280cc-ed98-4857-a90d-aef277d7bdaf-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:25:44.606431 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44604b33-ee78-40cd-bb55-e8c90d9cfcdf tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:45.161376 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44604b33-ee78-40cd-bb55-e8c90d9cfcdf tempest-VolumesBackupsTest-353079186 None] Volume volume-bb8280cc-ed98-4857-a90d-aef277d7bdaf is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:25:45.227775 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44604b33-ee78-40cd-bb55-e8c90d9cfcdf tempest-VolumesBackupsTest-353079186 None] deleting rbd volume volume-bb8280cc-ed98-4857-a90d-aef277d7bdaf {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:25:45.753133 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-44604b33-ee78-40cd-bb55-e8c90d9cfcdf tempest-VolumesBackupsTest-353079186 None] Created reservations ['b8b3ad49-3bdf-421a-908b-fd10747e7628', '729e8124-92de-40d9-b075-a413522d1206', 'cd14597a-6da4-4fc9-b1bb-95d2c8eb6138', '31e36900-78fc-464f-8cdb-9fec67cd8677'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:25:45.924214 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44604b33-ee78-40cd-bb55-e8c90d9cfcdf tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:46.045548 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44604b33-ee78-40cd-bb55-e8c90d9cfcdf tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:46.139965 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-44604b33-ee78-40cd-bb55-e8c90d9cfcdf tempest-VolumesBackupsTest-353079186 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:25:46.150972 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-44604b33-ee78-40cd-bb55-e8c90d9cfcdf tempest-VolumesBackupsTest-353079186 None] Deleted volume successfully. Sep 27 10:25:46.166962 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-44604b33-ee78-40cd-bb55-e8c90d9cfcdf tempest-VolumesBackupsTest-353079186 None] Lock "cinder-bb8280cc-ed98-4857-a90d-aef277d7bdaf-delete_volume" released by "delete_volume" :: held 1.640s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:25:48.100804 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:25:48.129129 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Flow 'volume_create_manager' (abd5b629-b5fb-4cfe-84f8-7dcd531da4d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:25:48.157444 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1abe1dcc-b181-48fb-863e-03f60f7129ed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:48.200116 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1abe1dcc-b181-48fb-863e-03f60f7129ed) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:25:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1592645171',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=13f3bdc5-42de-4f89-8c97-8fd979fd09c4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='430555edbc87497db7d7f79f2b891c04',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:25:48Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:25:48Z,use_quota=True,user_id='fbcfed06aa6b4177a4287064884125b2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:48.217777 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8915f41d-f7d6-4573-8cbc-359c6ef076a1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:48.231717 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8915f41d-f7d6-4573-8cbc-359c6ef076a1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:48.247656 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4a7e2521-7c1a-4346-b90b-71f118ea377c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:48.434083 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4a7e2521-7c1a-4346-b90b-71f118ea377c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '13f3bdc5-42de-4f89-8c97-8fd979fd09c4', 'volume_name': 'volume-13f3bdc5-42de-4f89-8c97-8fd979fd09c4', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:48.439695 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9deb078f-b351-4023-aebf-5b2c5718fb38) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:48.444907 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9deb078f-b351-4023-aebf-5b2c5718fb38) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:48.449666 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (88c3d045-6050-4096-a67d-46527306cb0d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:48.451098 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Volume 13f3bdc5-42de-4f89-8c97-8fd979fd09c4: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-13f3bdc5-42de-4f89-8c97-8fd979fd09c4', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:25:48.451853 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Cloning 13f3bdc5-42de-4f89-8c97-8fd979fd09c4 from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:25:48.453654 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:25:48.465715 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.012s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:25:48.481073 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 25, 48, 471875)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:25:48.482012 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:25:48.482659 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:25:48.488622 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.023s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:25:48.491344 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:25:48.501894 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 25, 48, 497750)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:25:48.502612 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:25:48.503202 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:25:48.522970 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:48.640831 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] creating snapshot='volume-13f3bdc5-42de-4f89-8c97-8fd979fd09c4.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:25:48.984603 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-13f3bdc5-42de-4f89-8c97-8fd979fd09c4.clone_snap' to 'volume-13f3bdc5-42de-4f89-8c97-8fd979fd09c4' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:25:49.214410 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:25:49.215954 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:25:49.216507 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:25:49.253059 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:25:49.254078 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Marking volume 13f3bdc5-42de-4f89-8c97-8fd979fd09c4 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:25:49.270039 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 13f3bdc5-42de-4f89-8c97-8fd979fd09c4. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:25:49.270898 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Creating volume glance metadata for volume 13f3bdc5-42de-4f89-8c97-8fd979fd09c4 backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:25:49.378338 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (88c3d045-6050-4096-a67d-46527306cb0d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-13f3bdc5-42de-4f89-8c97-8fd979fd09c4', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:49.391293 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (74f4837a-e2a6-4fa6-b0eb-10a308c83683) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:25:49.409179 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Volume volume-13f3bdc5-42de-4f89-8c97-8fd979fd09c4 (13f3bdc5-42de-4f89-8c97-8fd979fd09c4): created successfully Sep 27 10:25:49.415150 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (74f4837a-e2a6-4fa6-b0eb-10a308c83683) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:25:49.440757 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Flow 'volume_create_manager' (abd5b629-b5fb-4cfe-84f8-7dcd531da4d8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:25:49.470449 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6026db3d-c6ef-451a-94f6-2b66c885c889 tempest-VolumesBackupsTest-353079186 None] Created volume successfully. Sep 27 10:25:50.264821 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ca501896-ae72-43b2-ad5d-a6f0b5d64225 tempest-VolumesBackupsTest-353079186 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-27T10:25:50Z,data_timestamp=2022-09-27T10:25:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-1088979692',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=6eea6185-1821-4085-9642-dbe6cbb2750f,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='430555edbc87497db7d7f79f2b891c04',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='fbcfed06aa6b4177a4287064884125b2',volume_id=13f3bdc5-42de-4f89-8c97-8fd979fd09c4) Sep 27 10:25:50.266491 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.backup.rpcapi [None req-ca501896-ae72-43b2-ad5d-a6f0b5d64225 tempest-VolumesBackupsTest-353079186 None] continue_backup in rpcapi backup_id 6eea6185-1821-4085-9642-dbe6cbb2750f {{(pid=138473) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 27 10:25:50.403438 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-ca501896-ae72-43b2-ad5d-a6f0b5d64225 tempest-VolumesBackupsTest-353079186 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:25:51.629323 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-ca501896-ae72-43b2-ad5d-a6f0b5d64225 tempest-VolumesBackupsTest-353079186 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.225s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:25:51.631582 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ca501896-ae72-43b2-ad5d-a6f0b5d64225 tempest-VolumesBackupsTest-353079186 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-13f3bdc5-42de-4f89-8c97-8fd979fd09c4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '13f3bdc5-42de-4f89-8c97-8fd979fd09c4', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:25:51.666227 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ca501896-ae72-43b2-ad5d-a6f0b5d64225 tempest-VolumesBackupsTest-353079186 None] Initialize volume connection completed successfully. Sep 27 10:25:55.103535 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-aa6427fb-3b51-471e-a5d7-a25f612b16e9 tempest-TestVolumeBootPattern-4600220 None] Terminate volume connection completed successfully. Sep 27 10:25:55.106917 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-aa6427fb-3b51-471e-a5d7-a25f612b16e9 tempest-TestVolumeBootPattern-4600220 None] Deleting attachment 36589d4b-1385-437d-833e-3af319d83d7c. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:25:55.346929 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-4783613a-7994-4c89-b934-741d4e78edc5 tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-c0f96bfe-dd85-464f-97d3-93728dd4ca02-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:25:55.371485 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-4783613a-7994-4c89-b934-741d4e78edc5 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c0f96bfe-dd85-464f-97d3-93728dd4ca02-delete_volume" acquired by "delete_volume" :: waited 0.025s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:25:55.423820 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-4783613a-7994-4c89-b934-741d4e78edc5 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:55.858825 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-4783613a-7994-4c89-b934-741d4e78edc5 tempest-TestVolumeBootPattern-4600220 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:25:55.916587 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-4783613a-7994-4c89-b934-741d4e78edc5 tempest-TestVolumeBootPattern-4600220 None] deleting rbd volume volume-c0f96bfe-dd85-464f-97d3-93728dd4ca02 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:25:57.619968 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-4783613a-7994-4c89-b934-741d4e78edc5 tempest-TestVolumeBootPattern-4600220 None] Created reservations ['1c0e73d6-3984-4416-bd07-047506385ca7', 'caaf6949-9f4a-4c96-b726-d532005e2ceb', 'd0d6998b-03c2-4d52-9a0f-1eb0fded775a', '91ad7893-89b9-4e0b-92b6-d35596ba74f3'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:25:57.713071 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=51,cinder:INSERT=16,cinder:UPDATE=31,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:57.748323 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-4783613a-7994-4c89-b934-741d4e78edc5 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:57.830512 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-4783613a-7994-4c89-b934-741d4e78edc5 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:25:57.918849 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-4783613a-7994-4c89-b934-741d4e78edc5 tempest-TestVolumeBootPattern-4600220 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:25:57.925692 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-4783613a-7994-4c89-b934-741d4e78edc5 tempest-TestVolumeBootPattern-4600220 None] Deleted volume successfully. Sep 27 10:25:57.966979 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-336c664f-90da-467d-951a-2db0b0a4bf97 req-4783613a-7994-4c89-b934-741d4e78edc5 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c0f96bfe-dd85-464f-97d3-93728dd4ca02-delete_volume" released by "delete_volume" :: held 2.595s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:25:58.478602 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-76a659a4-73c3-48ed-bb50-9e410684bf3d tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-b3c99c48-92e3-4590-aff4-92ad11f3158c-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:25:58.478602 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-76a659a4-73c3-48ed-bb50-9e410684bf3d tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-b3c99c48-92e3-4590-aff4-92ad11f3158c-delete_snapshot" acquired by "delete_snapshot" :: waited 0.016s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:25:58.478602 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-76a659a4-73c3-48ed-bb50-9e410684bf3d tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:01.314309 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ca501896-ae72-43b2-ad5d-a6f0b5d64225 tempest-VolumesBackupsTest-353079186 None] Terminate volume connection completed successfully. Sep 27 10:26:01.343811 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ca501896-ae72-43b2-ad5d-a6f0b5d64225 tempest-VolumesBackupsTest-353079186 None] Remove volume export completed successfully. Sep 27 10:26:02.402850 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:26:02.427108 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Flow 'volume_create_manager' (3b22ffe9-0773-455f-af38-4e9bbfa1dcee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:02.434215 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2358c112-3bc1-4c8b-8db7-ae097179ca07) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:02.452468 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2358c112-3bc1-4c8b-8db7-ae097179ca07) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:26:02Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_6eea6185-1821-4085-9642-dbe6cbb2750f',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1c857eda-3be3-40f3-a721-b466616ea327,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='430555edbc87497db7d7f79f2b891c04',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:26:02Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:26:02Z,use_quota=True,user_id='fbcfed06aa6b4177a4287064884125b2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:02.458310 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c4208b96-e213-4deb-863f-2bf5484941c4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:02.463356 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c4208b96-e213-4deb-863f-2bf5484941c4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:02.468372 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f69b3f2d-d164-417e-9d6d-2b34de7b2953) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:02.472501 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f69b3f2d-d164-417e-9d6d-2b34de7b2953) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1c857eda-3be3-40f3-a721-b466616ea327', 'volume_name': 'volume-1c857eda-3be3-40f3-a721-b466616ea327', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:02.477588 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ecf42938-5029-4c95-b300-f1bc1595beb1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:02.481606 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ecf42938-5029-4c95-b300-f1bc1595beb1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:02.486649 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a03dfd3a-d8d0-456c-ad1b-c1705e6facda) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:02.489311 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Volume 1c857eda-3be3-40f3-a721-b466616ea327: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1c857eda-3be3-40f3-a721-b466616ea327', 'volume_size': 1} Sep 27 10:26:02.489715 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] creating volume 'volume-1c857eda-3be3-40f3-a721-b466616ea327' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:26:02.490504 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:02.604960 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:26:02.605338 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:26:02.609611 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a03dfd3a-d8d0-456c-ad1b-c1705e6facda) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1c857eda-3be3-40f3-a721-b466616ea327', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:02.616869 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c14d08f5-5db7-45e8-bd60-de54b84e44e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:02.635154 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Volume volume-1c857eda-3be3-40f3-a721-b466616ea327 (1c857eda-3be3-40f3-a721-b466616ea327): created successfully Sep 27 10:26:02.639268 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c14d08f5-5db7-45e8-bd60-de54b84e44e5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:02.644714 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Flow 'volume_create_manager' (3b22ffe9-0773-455f-af38-4e9bbfa1dcee) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:02.656303 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Created volume successfully. Sep 27 10:26:03.460651 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:26:04.461865 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.998s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:26:04.461865 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1c857eda-3be3-40f3-a721-b466616ea327', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '1c857eda-3be3-40f3-a721-b466616ea327', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:26:04.486119 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Initialize volume connection completed successfully. Sep 27 10:26:08.148441 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=59,cinder:UPDATE=16,cinder:INSERT=7,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:08.197624 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-0754ed73-1d74-48d7-a293-bd48b8c64aa4 tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['f9eb59f5-6385-4000-a810-08f0ef6a55ae', 'c926e798-62f1-48c3-8ab3-96aba949db1c', '587123c9-4e28-499a-a022-794c33fa143e', '4302d7fc-4ce4-4a55-9435-f8c2a3251116'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:08.273650 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0754ed73-1d74-48d7-a293-bd48b8c64aa4 tempest-VolumesSnapshotTestJSON-1705236581 None] Delete snapshot completed successfully. Sep 27 10:26:08.290436 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0754ed73-1d74-48d7-a293-bd48b8c64aa4 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-ce92b4af-4c17-43ea-af12-e6ccaf22b49a-delete_snapshot" released by "delete_snapshot" :: held 30.402s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:08.563653 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-2ef7269a-5830-417d-b97a-6080d84ad12c tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-eee4168a-482d-499d-9e0e-a3f76b39f2d4-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:08.584562 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-2ef7269a-5830-417d-b97a-6080d84ad12c tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-eee4168a-482d-499d-9e0e-a3f76b39f2d4-delete_snapshot" acquired by "delete_snapshot" :: waited 0.020s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:08.594638 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2ef7269a-5830-417d-b97a-6080d84ad12c tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:10.629830 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Terminate volume connection completed successfully. Sep 27 10:26:10.652012 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-66d3a057-134e-42ca-8c7a-590bc1b9df62 tempest-VolumesBackupsTest-353079186 None] Remove volume export completed successfully. Sep 27 10:26:12.009965 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-304ff36e-e802-430b-bd48-d7cc727ee26c tempest-VolumesBackupsTest-353079186 None] Acquiring lock "cinder-1c857eda-3be3-40f3-a721-b466616ea327-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:12.033056 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-304ff36e-e802-430b-bd48-d7cc727ee26c tempest-VolumesBackupsTest-353079186 None] Lock "cinder-1c857eda-3be3-40f3-a721-b466616ea327-delete_volume" acquired by "delete_volume" :: waited 0.025s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:12.109145 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-304ff36e-e802-430b-bd48-d7cc727ee26c tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:12.978222 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-304ff36e-e802-430b-bd48-d7cc727ee26c tempest-VolumesBackupsTest-353079186 None] Volume volume-1c857eda-3be3-40f3-a721-b466616ea327 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:26:12.997189 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-304ff36e-e802-430b-bd48-d7cc727ee26c tempest-VolumesBackupsTest-353079186 None] deleting rbd volume volume-1c857eda-3be3-40f3-a721-b466616ea327 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:26:13.526323 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-304ff36e-e802-430b-bd48-d7cc727ee26c tempest-VolumesBackupsTest-353079186 None] Created reservations ['776baae3-2faa-406c-8d22-8cb184a75fec', '89ee4366-1d30-46b5-88a1-8ab76d8f69bc', '008af5d9-f018-4b50-994a-8518593b0e10', 'a3155fc1-f2a3-425d-8643-a317ad103c92'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:13.629568 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-304ff36e-e802-430b-bd48-d7cc727ee26c tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:13.697144 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-304ff36e-e802-430b-bd48-d7cc727ee26c tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:13.738867 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-304ff36e-e802-430b-bd48-d7cc727ee26c tempest-VolumesBackupsTest-353079186 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:26:13.745184 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-304ff36e-e802-430b-bd48-d7cc727ee26c tempest-VolumesBackupsTest-353079186 None] Deleted volume successfully. Sep 27 10:26:13.767139 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-304ff36e-e802-430b-bd48-d7cc727ee26c tempest-VolumesBackupsTest-353079186 None] Lock "cinder-1c857eda-3be3-40f3-a721-b466616ea327-delete_volume" released by "delete_volume" :: held 1.734s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:16.446051 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] Acquiring lock "cinder-13f3bdc5-42de-4f89-8c97-8fd979fd09c4-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:16.460608 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] Lock "cinder-13f3bdc5-42de-4f89-8c97-8fd979fd09c4-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:16.510345 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:17.197182 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] deleting rbd volume volume-13f3bdc5-42de-4f89-8c97-8fd979fd09c4 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:26:17.510868 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:26:17.569511 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] deleting parent snapshot volume-13f3bdc5-42de-4f89-8c97-8fd979fd09c4.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:26:18.292307 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] Created reservations ['1986ed71-8b9d-4425-a797-5029a1b44cd0', '12d7bbd7-b9f1-4af0-b0d4-d94291dc7d4a', 'bb4af354-6a32-4f3a-9da0-ae9524e17a74', 'e5042c6a-b245-48a9-9c62-822c096e7691'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:18.374551 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:18.431740 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:18.473620 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:26:18.482965 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] Deleted volume successfully. Sep 27 10:26:18.506889 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-52d63afa-9b6f-49ff-bb0f-d90cbce11102 tempest-VolumesBackupsTest-353079186 None] Lock "cinder-13f3bdc5-42de-4f89-8c97-8fd979fd09c4-delete_volume" released by "delete_volume" :: held 2.046s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:21.171939 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:26:21.199876 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Flow 'volume_create_manager' (12e4fe0b-5819-42f4-80f1-6a75a0c78a92) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:21.206848 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8da38953-7eed-485f-a8bd-754177084aa8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:21.228209 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8da38953-7eed-485f-a8bd-754177084aa8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:26:21Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-425590472',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='08ded570affb4de69a0d6ffbf0ea80aa',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:26:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:26:21Z,use_quota=True,user_id='ff7c05e217c4401eb103a80188392d64',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:21.233721 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (030462a8-7bd6-49d5-8e85-c5a3371e39cd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:21.237885 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (030462a8-7bd6-49d5-8e85-c5a3371e39cd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:21.243924 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (74c02ee5-d4ff-473c-b5c2-efdf578b563f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:21.249043 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (74c02ee5-d4ff-473c-b5c2-efdf578b563f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2', 'volume_name': 'volume-b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:21.254525 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a63cd859-c3e0-4ed3-8658-5f871347fafe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:21.260381 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a63cd859-c3e0-4ed3-8658-5f871347fafe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:21.265155 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (db1a2248-f198-4cd9-9fb5-a463939841e9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:21.266234 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Volume b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2', 'volume_size': 1} Sep 27 10:26:21.266632 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] creating volume 'volume-b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:26:21.267413 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:21.329764 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:26:21.330393 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:26:21.334349 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (db1a2248-f198-4cd9-9fb5-a463939841e9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:21.339489 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e55ed0be-de50-487b-8230-7f0907290961) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:21.351107 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Volume volume-b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2 (b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2): created successfully Sep 27 10:26:21.355081 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e55ed0be-de50-487b-8230-7f0907290961) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:21.360727 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Flow 'volume_create_manager' (12e4fe0b-5819-42f4-80f1-6a75a0c78a92) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:21.371367 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-f2844fc6-71b0-4306-9679-3b0136185566 tempest-VolumesListTestJSON-1325555620 None] Created volume successfully. Sep 27 10:26:22.655099 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:26:22.676059 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Flow 'volume_create_manager' (396bac64-ba5f-4782-b020-a7ed2d50b8dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:22.682434 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (485d12a0-a175-4d88-9e20-ec19456445b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:22.687629 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=67,cinder:UPDATE=25,cinder:DELETE=2,cinder:INSERT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:22.699660 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (485d12a0-a175-4d88-9e20-ec19456445b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:26:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-3950075',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=41c56e1c-2d0b-42e4-89c0-81921fa7512e,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='08ded570affb4de69a0d6ffbf0ea80aa',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:26:23Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:26:23Z,use_quota=True,user_id='ff7c05e217c4401eb103a80188392d64',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:22.704773 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (06c2f4a4-d3a9-41db-b530-21b964ae34a0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:22.708886 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (06c2f4a4-d3a9-41db-b530-21b964ae34a0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:22.714696 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0f4b9bc8-6fce-40bb-924e-5fada65bf8c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:22.718809 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0f4b9bc8-6fce-40bb-924e-5fada65bf8c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '41c56e1c-2d0b-42e4-89c0-81921fa7512e', 'volume_name': 'volume-41c56e1c-2d0b-42e4-89c0-81921fa7512e', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:22.724570 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3b720914-d996-4f21-9493-11dc616d03a2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:22.728562 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3b720914-d996-4f21-9493-11dc616d03a2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:22.733015 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1de9bb58-1dc0-446d-a4de-2a7c1d5ac4a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:22.734170 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Volume 41c56e1c-2d0b-42e4-89c0-81921fa7512e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-41c56e1c-2d0b-42e4-89c0-81921fa7512e', 'volume_size': 1} Sep 27 10:26:22.734558 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] creating volume 'volume-41c56e1c-2d0b-42e4-89c0-81921fa7512e' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:26:22.735205 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:22.789015 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:26:22.789383 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:26:22.793075 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1de9bb58-1dc0-446d-a4de-2a7c1d5ac4a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-41c56e1c-2d0b-42e4-89c0-81921fa7512e', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:22.797654 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2b573af1-1001-4dd6-af83-029b6a39e6ae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:22.807911 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Volume volume-41c56e1c-2d0b-42e4-89c0-81921fa7512e (41c56e1c-2d0b-42e4-89c0-81921fa7512e): created successfully Sep 27 10:26:22.811568 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2b573af1-1001-4dd6-af83-029b6a39e6ae) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:22.817043 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Flow 'volume_create_manager' (396bac64-ba5f-4782-b020-a7ed2d50b8dd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:22.826760 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ec3e2c8e-be98-4f82-a443-42d5261a4ff0 tempest-VolumesListTestJSON-1325555620 None] Created volume successfully. Sep 27 10:26:24.138029 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:26:24.160649 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Flow 'volume_create_manager' (3b4310e9-a6ff-4538-83de-60d1d7d73ff4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:24.167945 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7e7c7dba-f68a-4914-88f5-fdbabebcea44) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:24.193704 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7e7c7dba-f68a-4914-88f5-fdbabebcea44) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:26:24Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-21740101',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c9f62936-b136-4e33-afa6-38d8e8859399,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='08ded570affb4de69a0d6ffbf0ea80aa',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:26:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:26:24Z,use_quota=True,user_id='ff7c05e217c4401eb103a80188392d64',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:24.199680 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e47ce11d-6a3d-4ce5-8e78-fa902d57e0f4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:24.203734 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e47ce11d-6a3d-4ce5-8e78-fa902d57e0f4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:24.208967 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (151ada56-c62e-49ff-8ee8-65b24a78828c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:24.213645 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (151ada56-c62e-49ff-8ee8-65b24a78828c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c9f62936-b136-4e33-afa6-38d8e8859399', 'volume_name': 'volume-c9f62936-b136-4e33-afa6-38d8e8859399', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:24.218741 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b75257bd-fcb0-49d8-99bf-ff81feb04c23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:24.222816 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b75257bd-fcb0-49d8-99bf-ff81feb04c23) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:24.227590 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (56a268d9-8b21-4d8c-9b4a-7b94e04df298) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:24.228880 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Volume c9f62936-b136-4e33-afa6-38d8e8859399: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c9f62936-b136-4e33-afa6-38d8e8859399', 'volume_size': 1} Sep 27 10:26:24.229278 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] creating volume 'volume-c9f62936-b136-4e33-afa6-38d8e8859399' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:26:24.229955 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:24.293548 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:26:24.294008 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:26:24.298217 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (56a268d9-8b21-4d8c-9b4a-7b94e04df298) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c9f62936-b136-4e33-afa6-38d8e8859399', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:24.304154 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8f28a7d6-e79a-4c3e-bc6b-09bd69624a36) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:24.315249 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Volume volume-c9f62936-b136-4e33-afa6-38d8e8859399 (c9f62936-b136-4e33-afa6-38d8e8859399): created successfully Sep 27 10:26:24.321064 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8f28a7d6-e79a-4c3e-bc6b-09bd69624a36) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:24.328154 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Flow 'volume_create_manager' (3b4310e9-a6ff-4538-83de-60d1d7d73ff4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:24.341364 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-72ce9bcc-b9b2-449b-a67b-66b0226e71ae tempest-VolumesListTestJSON-1325555620 None] Created volume successfully. Sep 27 10:26:27.846671 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] Acquiring lock "cinder-c9f62936-b136-4e33-afa6-38d8e8859399-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:27.858655 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] Lock "cinder-c9f62936-b136-4e33-afa6-38d8e8859399-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:27.918396 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:27.989410 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:26:27.990086 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] Volume volume-c9f62936-b136-4e33-afa6-38d8e8859399 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:26:28.002416 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] deleting rbd volume volume-c9f62936-b136-4e33-afa6-38d8e8859399 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:26:28.346343 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] Created reservations ['e1ef61f2-f21a-4db0-a5e7-58ac5aae3bf1', 'af5ed5b3-b606-41d2-b40b-a34b401100ae', '21449ac0-3545-40c4-9c95-bfce59945fa6', '5d03aa9a-6ee8-41fb-aa32-4b52172a05af'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:28.416598 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:28.465704 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:28.518344 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:26:28.526294 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] Deleted volume successfully. Sep 27 10:26:28.547087 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-95512b84-8ec8-4c90-b213-96e323182369 tempest-VolumesListTestJSON-1325555620 None] Lock "cinder-c9f62936-b136-4e33-afa6-38d8e8859399-delete_volume" released by "delete_volume" :: held 0.688s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:29.048956 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] Acquiring lock "cinder-41c56e1c-2d0b-42e4-89c0-81921fa7512e-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:29.074601 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] Lock "cinder-41c56e1c-2d0b-42e4-89c0-81921fa7512e-delete_volume" acquired by "delete_volume" :: waited 0.026s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:29.139252 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:INSERT=7,cinder:UPDATE=36,cinder:SELECT=54,cinder:DELETE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:29.140512 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:29.225407 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:26:29.225859 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] Volume volume-41c56e1c-2d0b-42e4-89c0-81921fa7512e is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:26:29.241096 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] deleting rbd volume volume-41c56e1c-2d0b-42e4-89c0-81921fa7512e {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:26:29.425423 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-76a659a4-73c3-48ed-bb50-9e410684bf3d tempest-TestVolumeBootPattern-4600220 None] Created reservations ['f5a3cdbb-2053-45aa-8f34-77bb52b85f58', '65f9bb2d-f1f9-4e21-860c-2a1aa7916fcd', '538a2632-9a8b-495d-9167-f789002322f9', '486f2615-6acf-4281-a6ca-f732c75b7eab'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:29.500827 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-76a659a4-73c3-48ed-bb50-9e410684bf3d tempest-TestVolumeBootPattern-4600220 None] Delete snapshot completed successfully. Sep 27 10:26:29.506968 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-76a659a4-73c3-48ed-bb50-9e410684bf3d tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-b3c99c48-92e3-4590-aff4-92ad11f3158c-delete_snapshot" released by "delete_snapshot" :: held 31.036s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:29.615084 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] Created reservations ['f94bb359-03a8-4e3b-b0e7-df87e61c7b3d', '5c55e602-312e-4cec-a8a1-7dd388b38e24', '2476e77c-41b1-41f8-9e1e-ffe42bec7176', 'caf3ce8e-1e82-4041-b4d9-49ce0ec14fea'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:29.726488 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:29.790306 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:29.854663 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:26:29.859296 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] Deleted volume successfully. Sep 27 10:26:29.875716 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-dda482d1-47fe-4974-90dc-7cd12d1b02fb tempest-VolumesListTestJSON-1325555620 None] Lock "cinder-41c56e1c-2d0b-42e4-89c0-81921fa7512e-delete_volume" released by "delete_volume" :: held 0.801s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:30.267199 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] Acquiring lock "cinder-b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:30.285631 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] Lock "cinder-b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:30.332518 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:30.399984 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:26:30.400874 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] Volume volume-b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:26:30.408752 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] deleting rbd volume volume-b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:26:30.719876 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] Created reservations ['38d36f57-0ce1-4d25-9d4b-da18f687a9d7', '94873af8-87df-49d2-a424-8da6084612e1', 'd27176ab-8596-4e17-b41d-f7d4f5fbb553', 'a2a8a604-fa75-480d-94a1-d7dfa261e8d1'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:30.761848 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=55,cinder:UPDATE=33,cinder:DELETE=3,cinder:INSERT=9 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:30.785014 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:30.852432 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:30.913314 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:26:30.919433 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] Deleted volume successfully. Sep 27 10:26:30.934738 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-f1dafe11-0bd2-4394-a1fc-a7e69949ce05 tempest-VolumesListTestJSON-1325555620 None] Lock "cinder-b1f2a157-8dd9-4a18-9e7d-eeeda858e1b2-delete_volume" released by "delete_volume" :: held 0.649s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:32.679539 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-e7365eb6-5b80-4a44-b592-4a4c4dea44e2 tempest-TestVolumeBootPattern-4600220 None] Terminate volume connection completed successfully. Sep 27 10:26:32.680257 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-e7365eb6-5b80-4a44-b592-4a4c4dea44e2 tempest-TestVolumeBootPattern-4600220 None] Deleting attachment 3ca665a8-fa80-4866-8bd9-f57a4868eb01. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:26:32.847428 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-c2bfd580-e62a-4f33-a8ea-3e0437752d94-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:32.861607 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c2bfd580-e62a-4f33-a8ea-3e0437752d94-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:32.936972 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:33.108158 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:26:33.137632 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] deleting rbd volume volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:26:33.524412 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:26:33.574915 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] deleting parent snapshot volume-c2bfd580-e62a-4f33-a8ea-3e0437752d94.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:26:34.351805 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] Created reservations ['33870f78-1763-4e4c-bd43-4c48356e699d', '448312a7-2db5-4a43-8801-722330ab6541', 'dcac1415-f0fd-4f5a-b98c-20077a75680f', '790327ee-d0dd-4764-9f2c-2d31ccf6ecef'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:34.462708 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:34.541050 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:34.613189 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:26:34.617889 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] Deleted volume successfully. Sep 27 10:26:34.639792 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-e152c28f-e7e1-4955-9ceb-4d92833cb03d req-07b740db-ff56-4b74-b226-41f876d94159 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c2bfd580-e62a-4f33-a8ea-3e0437752d94-delete_volume" released by "delete_volume" :: held 1.778s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:39.007091 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-2ef7269a-5830-417d-b97a-6080d84ad12c tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['3d92122a-3bfd-44c5-9446-fab30eee1494', '2807c379-e5ad-48d5-96b6-1ad78956aa46', '44e05af6-54c3-450e-b2ad-db74eb9f67e8', '9296967b-4e9f-437d-94a2-bc9e892214f4'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:39.078558 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=53,cinder:INSERT=11,cinder:UPDATE=34,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:39.096309 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-2ef7269a-5830-417d-b97a-6080d84ad12c tempest-VolumesSnapshotTestJSON-1705236581 None] Delete snapshot completed successfully. Sep 27 10:26:39.111724 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-2ef7269a-5830-417d-b97a-6080d84ad12c tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-eee4168a-482d-499d-9e0e-a3f76b39f2d4-delete_snapshot" released by "delete_snapshot" :: held 30.527s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:39.513434 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:26:39.549296 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (d9c2bcdd-87c6-45b9-8d5a-939c8097af68) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:39.560669 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6e7c5a81-edd1-4e43-a9e9-6d0dd1a29e18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:39.583317 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6e7c5a81-edd1-4e43-a9e9-6d0dd1a29e18) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:26:39Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1736078724',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8489f678-48b9-47ca-8f5e-397c453365e4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='64f8fe93425d4e97b1ae5661172f4846',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:26:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:26:39Z,use_quota=True,user_id='178956f6c8154a06a0710b295aff3086',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:39.591960 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f914acec-cfad-43ec-839a-eb9926756f9f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:39.598887 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f914acec-cfad-43ec-839a-eb9926756f9f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:39.609830 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bb540cb1-17db-41b4-a3e7-62cb15b722c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:39.750159 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bb540cb1-17db-41b4-a3e7-62cb15b722c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '8489f678-48b9-47ca-8f5e-397c453365e4', 'volume_name': 'volume-8489f678-48b9-47ca-8f5e-397c453365e4', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:39.762934 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ad76c892-2fe7-47ad-902a-1013fbb98d18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:39.773427 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ad76c892-2fe7-47ad-902a-1013fbb98d18) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:39.780048 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f1fe9fca-94ab-4899-8536-6b4f1f9a2962) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:39.783575 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Volume 8489f678-48b9-47ca-8f5e-397c453365e4: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-8489f678-48b9-47ca-8f5e-397c453365e4', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:26:39.783575 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Cloning 8489f678-48b9-47ca-8f5e-397c453365e4 from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:26:39.784567 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:39.804377 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.018s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:39.817837 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 26, 39, 808539)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:26:39.817837 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:26:39.818722 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:26:39.826529 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.023s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:39.827797 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:26:39.838911 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 26, 39, 833098)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:26:39.839627 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:26:39.840180 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:26:39.854806 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:39.957897 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] creating snapshot='volume-8489f678-48b9-47ca-8f5e-397c453365e4.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:26:40.132999 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-8489f678-48b9-47ca-8f5e-397c453365e4.clone_snap' to 'volume-8489f678-48b9-47ca-8f5e-397c453365e4' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:26:40.348406 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:26:40.361614 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:26:40.363581 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:26:40.391762 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:26:40.391762 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Marking volume 8489f678-48b9-47ca-8f5e-397c453365e4 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:26:40.401324 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] Acquiring lock "cinder-d558400b-8728-4133-8e2d-d76be2e106d8-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:40.415746 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] Lock "cinder-d558400b-8728-4133-8e2d-d76be2e106d8-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:40.421059 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 8489f678-48b9-47ca-8f5e-397c453365e4. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:26:40.421673 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Creating volume glance metadata for volume 8489f678-48b9-47ca-8f5e-397c453365e4 backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:26:40.513461 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:40.578415 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:26:40.578837 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] Volume volume-d558400b-8728-4133-8e2d-d76be2e106d8 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:26:40.591630 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] deleting rbd volume volume-d558400b-8728-4133-8e2d-d76be2e106d8 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:26:40.664412 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f1fe9fca-94ab-4899-8536-6b4f1f9a2962) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8489f678-48b9-47ca-8f5e-397c453365e4', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:40.678107 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bb522258-9e88-41f7-a57f-be0d4947e2d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:40.727295 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Volume volume-8489f678-48b9-47ca-8f5e-397c453365e4 (8489f678-48b9-47ca-8f5e-397c453365e4): created successfully Sep 27 10:26:40.740324 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bb522258-9e88-41f7-a57f-be0d4947e2d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:40.754678 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (d9c2bcdd-87c6-45b9-8d5a-939c8097af68) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:40.767981 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-dfa8e6cb-5ba8-4359-aba0-fa86a2f048e6 tempest-TestVolumeBootPattern-4600220 None] Created volume successfully. Sep 27 10:26:40.985578 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] Created reservations ['d229915f-7cff-4947-9203-5586691a0598', '240c8eaa-4858-432e-a9a1-2c8bc2619d8b', '776a6b51-3225-4aa7-aaa9-08638e553e00', 'd4273dc9-2b53-4c44-9911-dd84f9cbd842'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:41.057871 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:41.095099 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:41.142521 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:26:41.147127 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] Deleted volume successfully. Sep 27 10:26:41.163856 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-203e493c-3e46-48f9-baf7-793da2be4f2d tempest-AttachVolumeShelveTestJSON-1912240963 None] Lock "cinder-d558400b-8728-4133-8e2d-d76be2e106d8-delete_volume" released by "delete_volume" :: held 0.748s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:41.705253 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 req-1d476aef-8e96-4860-913c-0486ebb7edab tempest-VolumesSnapshotTestJSON-1705236581 None] Terminate volume connection completed successfully. Sep 27 10:26:41.706761 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 req-1d476aef-8e96-4860-913c-0486ebb7edab tempest-VolumesSnapshotTestJSON-1705236581 None] Deleting attachment 6830ae69-41cc-4e47-a6b1-d53f71736393. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:26:41.792210 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] Acquiring lock "cinder-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:41.811996 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] Lock "cinder-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d-delete_volume" acquired by "delete_volume" :: waited 0.020s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:41.874076 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:41.950740 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:26:41.951282 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] Volume volume-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:26:41.966429 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] deleting rbd volume volume-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:26:42.517038 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] Created reservations ['4dca1d87-8101-49e6-b6c9-540550068755', 'f22cfb1e-5ee7-46d5-bb9c-3233d71dbc5d', '74c06d31-cf9c-4314-bb72-5320177d3656', 'ecbcd835-0dc8-40a9-924b-442971439087'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:42.589174 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=63,cinder:INSERT=15,cinder:UPDATE=21,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:42.601339 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:42.678080 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:42.765656 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:26:42.775763 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] Deleted volume successfully. Sep 27 10:26:42.804995 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-44951aa6-f1ed-448d-b031-ac03b0bbe3b2 tempest-AttachVolumeShelveTestJSON-1912240963 None] Lock "cinder-05adfdbe-11b6-4e2e-9ace-45f0a99dba1d-delete_volume" released by "delete_volume" :: held 0.993s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:46.235563 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-18d07a7c-b409-4540-a6d3-709bb588c326 req-b0e17a53-94ca-438c-a1db-c6c826d5c4d1 tempest-TestVolumeBootPattern-4600220 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:26:46.674275 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3a0298e6-48d5-402a-9ac7-e109b3e51bd8 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:47.069234 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-18d07a7c-b409-4540-a6d3-709bb588c326 req-b0e17a53-94ca-438c-a1db-c6c826d5c4d1 tempest-TestVolumeBootPattern-4600220 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.834s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:26:47.069998 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-18d07a7c-b409-4540-a6d3-709bb588c326 req-b0e17a53-94ca-438c-a1db-c6c826d5c4d1 tempest-TestVolumeBootPattern-4600220 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-8489f678-48b9-47ca-8f5e-397c453365e4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '8489f678-48b9-47ca-8f5e-397c453365e4', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:26:47.094112 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-18d07a7c-b409-4540-a6d3-709bb588c326 req-b0e17a53-94ca-438c-a1db-c6c826d5c4d1 tempest-TestVolumeBootPattern-4600220 None] Connection info returned from driver {'name': 'volumes/volume-8489f678-48b9-47ca-8f5e-397c453365e4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '8489f678-48b9-47ca-8f5e-397c453365e4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '7d7112d2-60a1-4cff-8c78-03eecfe73a4f'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:26:47.136140 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=57,cinder:UPDATE=33,cinder:INSERT=8,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:47.150590 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-18d07a7c-b409-4540-a6d3-709bb588c326 req-b0e17a53-94ca-438c-a1db-c6c826d5c4d1 tempest-TestVolumeBootPattern-4600220 None] attachment_update completed successfully. Sep 27 10:26:47.322221 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-3a0298e6-48d5-402a-9ac7-e109b3e51bd8 tempest-VolumesSnapshotTestJSON-1705236581 None] Create snapshot completed successfully Sep 27 10:26:47.971850 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-88b3c892-9a8c-43cd-a40a-4fb8343ee1dd tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-91639a9f-c97a-46ce-8380-bd90d1f83db1-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:47.994551 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-88b3c892-9a8c-43cd-a40a-4fb8343ee1dd tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-91639a9f-c97a-46ce-8380-bd90d1f83db1-delete_snapshot" acquired by "delete_snapshot" :: waited 0.023s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:47.996861 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-88b3c892-9a8c-43cd-a40a-4fb8343ee1dd tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:49.267341 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-88b3c892-9a8c-43cd-a40a-4fb8343ee1dd tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['8e48706b-ec34-47e8-a8b9-330fd8817c5e', 'ec7e2717-900e-4abf-adfa-0e246e6ecb76', '9c4f1735-c5bb-4337-bf90-a81f90eb531a', '76c2a8cd-a38f-44ef-9180-c15dd39b81a6'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:49.347492 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-88b3c892-9a8c-43cd-a40a-4fb8343ee1dd tempest-VolumesSnapshotTestJSON-1705236581 None] Delete snapshot completed successfully. Sep 27 10:26:49.352981 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-88b3c892-9a8c-43cd-a40a-4fb8343ee1dd tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-91639a9f-c97a-46ce-8380-bd90d1f83db1-delete_snapshot" released by "delete_snapshot" :: held 1.358s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:26:50.621739 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-04dea918-b1f9-4723-80ee-d0c73a162203 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:51.532044 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-04dea918-b1f9-4723-80ee-d0c73a162203 tempest-VolumesSnapshotTestJSON-1705236581 None] Create snapshot completed successfully Sep 27 10:26:52.686016 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:26:52.709891 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Flow 'volume_create_manager' (ca03898e-cff2-4bd1-b9b7-9d94f9c6132f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:52.719244 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5795ffa5-9b11-4238-bd3b-fc2b2add36c9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:52.764930 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5795ffa5-9b11-4238-bd3b-fc2b2add36c9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:26:52Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeRetypeWithoutMigrationTest-Volume-124549192',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=13409778-b20e-4b96-99a1-b5140ed1d219,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3bf83c89117946088316cbd8f1de8fe2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:26:53Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:26:53Z,use_quota=True,user_id='e720b6bdc84b488f90165d8971f4198e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(85cc87ac-cfdf-46f0-b050-f0eae3976979),volume_type_id=85cc87ac-cfdf-46f0-b050-f0eae3976979)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:52.805081 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (238e5e69-9e66-4085-bb6d-8feb2fafcb5f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:52.816274 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (238e5e69-9e66-4085-bb6d-8feb2fafcb5f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:52.826995 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1783c20d-6e92-4a68-b2d4-430f7d1ffeaa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:52.841939 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1783c20d-6e92-4a68-b2d4-430f7d1ffeaa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '13409778-b20e-4b96-99a1-b5140ed1d219', 'volume_name': 'volume-13409778-b20e-4b96-99a1-b5140ed1d219', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:52.847542 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (57982058-a394-4bba-ad67-c15c169b5e74) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:52.853554 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (57982058-a394-4bba-ad67-c15c169b5e74) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:52.858815 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d01cae33-0734-4cc0-b4f4-c853aad8fc9b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:52.860058 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Volume 13409778-b20e-4b96-99a1-b5140ed1d219: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-13409778-b20e-4b96-99a1-b5140ed1d219', 'volume_size': 1} Sep 27 10:26:52.860716 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] creating volume 'volume-13409778-b20e-4b96-99a1-b5140ed1d219' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:26:52.861928 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:52.970425 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:26:52.970810 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:26:52.975577 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d01cae33-0734-4cc0-b4f4-c853aad8fc9b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-13409778-b20e-4b96-99a1-b5140ed1d219', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:52.981469 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (00848f5b-7edd-4634-89ec-a56633d4eabd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:26:53.023975 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Volume volume-13409778-b20e-4b96-99a1-b5140ed1d219 (13409778-b20e-4b96-99a1-b5140ed1d219): created successfully Sep 27 10:26:53.028561 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (00848f5b-7edd-4634-89ec-a56633d4eabd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:26:53.041718 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Flow 'volume_create_manager' (ca03898e-cff2-4bd1-b9b7-9d94f9c6132f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:26:53.073912 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5cff5513-2fc9-4be4-8c95-ca137d652a87 tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Created volume successfully. Sep 27 10:26:54.301529 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d9a7bcd4-1407-4d3c-9b47-3e0a46610d0d tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:54.388528 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d9a7bcd4-1407-4d3c-9b47-3e0a46610d0d tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:54.441006 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-d9a7bcd4-1407-4d3c-9b47-3e0a46610d0d tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:26:54.446256 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d9a7bcd4-1407-4d3c-9b47-3e0a46610d0d tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Retype volume completed successfully. Sep 27 10:26:55.642153 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Acquiring lock "cinder-13409778-b20e-4b96-99a1-b5140ed1d219-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:26:55.655401 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Lock "cinder-13409778-b20e-4b96-99a1-b5140ed1d219-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:26:55.764221 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:55.867957 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:26:55.869504 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Volume volume-13409778-b20e-4b96-99a1-b5140ed1d219 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:26:55.897498 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] deleting rbd volume volume-13409778-b20e-4b96-99a1-b5140ed1d219 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:26:56.253892 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Deprecated: Default quota for resource: volumes_volume-type-2 is set by the default quota flag: quota_volumes_volume-type-2, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:26:56.255389 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Deprecated: Default quota for resource: gigabytes_volume-type-2 is set by the default quota flag: quota_gigabytes_volume-type-2, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:26:56.306868 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Created reservations ['45edb9a6-f06e-4a41-a09d-84d32797cc67', '24fad255-ba06-4ec2-a1e4-db55427b75fe', 'b8d05715-0726-48b4-8553-89b950e72efd', '122a6b0d-a3fc-40ac-8ceb-8076083481d9'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:26:56.374236 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:56.459562 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:26:56.552380 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:26:56.558062 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Deleted volume successfully. Sep 27 10:26:56.579470 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8f04f2cc-e530-4ed9-881a-992ddb7f9f7b tempest-VolumeRetypeWithoutMigrationTest-1280588400 None] Lock "cinder-13409778-b20e-4b96-99a1-b5140ed1d219-delete_volume" released by "delete_volume" :: held 0.923s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:07.200519 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:07.378895 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Flow 'volume_create_manager' (ececd5ad-f95d-4ce2-88da-4baf76aa0eed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:07.395039 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (69efa79f-c401-456c-b6a6-acd0762d590a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:07.445441 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (69efa79f-c401-456c-b6a6-acd0762d590a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-109936636',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a3cf6aff-9473-4a81-8f1e-1034bda2a859,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='99e1389595b74807b615e1c6a289ca10',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:07Z,use_quota=True,user_id='5ac35bd9df384cfcadd179543142ba80',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:07.454581 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f308fc63-c9d4-4e15-8ed8-397189462e4f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:07.476849 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f308fc63-c9d4-4e15-8ed8-397189462e4f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:07.482200 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e1598b25-5b72-453c-8391-a8c08e10f5bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:07.495149 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e1598b25-5b72-453c-8391-a8c08e10f5bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a3cf6aff-9473-4a81-8f1e-1034bda2a859', 'volume_name': 'volume-a3cf6aff-9473-4a81-8f1e-1034bda2a859', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:07.507974 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a6d35384-dbe9-4a3e-9f38-fa353e28a0ce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:07.512396 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a6d35384-dbe9-4a3e-9f38-fa353e28a0ce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:07.517527 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fbff35a5-a6f2-4526-926d-2ec790b777c2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:07.518881 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Volume a3cf6aff-9473-4a81-8f1e-1034bda2a859: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a3cf6aff-9473-4a81-8f1e-1034bda2a859', 'volume_size': 1} Sep 27 10:27:07.519498 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] creating volume 'volume-a3cf6aff-9473-4a81-8f1e-1034bda2a859' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:27:07.520543 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:07.652317 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:07.652718 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:07.656799 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fbff35a5-a6f2-4526-926d-2ec790b777c2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a3cf6aff-9473-4a81-8f1e-1034bda2a859', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:07.661923 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (70c229a1-f6ac-43e0-8cb2-657a58f0b8da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:07.685404 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Volume volume-a3cf6aff-9473-4a81-8f1e-1034bda2a859 (a3cf6aff-9473-4a81-8f1e-1034bda2a859): created successfully Sep 27 10:27:07.697578 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (70c229a1-f6ac-43e0-8cb2-657a58f0b8da) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:07.713851 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Flow 'volume_create_manager' (ececd5ad-f95d-4ce2-88da-4baf76aa0eed) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:07.725749 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-9c4ca9f6-1c1e-4301-822e-e5da073044ea tempest-VolumesListAdminTestJSON-396610026 None] Created volume successfully. Sep 27 10:27:09.069469 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:09.120083 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Flow 'volume_create_manager' (b4ea0d21-db8f-46f8-88da-39bc4b808913) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:09.128708 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1bfe2485-c0cc-48b3-8acb-f55a5c6509b4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:09.166805 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1bfe2485-c0cc-48b3-8acb-f55a5c6509b4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-1463206030',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d51a3a0f-f047-4a0e-be70-00f264cc0c50,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='99e1389595b74807b615e1c6a289ca10',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:09Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:09Z,use_quota=True,user_id='5ac35bd9df384cfcadd179543142ba80',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:09.174675 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ee4f7915-f991-48d6-b4ba-b256f65b405d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:09.181027 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ee4f7915-f991-48d6-b4ba-b256f65b405d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:09.187087 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2f72d5f5-b979-438c-96d2-d2f449c9aa2a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:09.191667 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2f72d5f5-b979-438c-96d2-d2f449c9aa2a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd51a3a0f-f047-4a0e-be70-00f264cc0c50', 'volume_name': 'volume-d51a3a0f-f047-4a0e-be70-00f264cc0c50', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:09.197759 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a1f39334-6212-4841-8ee3-9e1b6d7b1476) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:09.202036 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a1f39334-6212-4841-8ee3-9e1b6d7b1476) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:09.207845 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (83d0d32f-4743-4d8e-9765-95ecb6418f50) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:09.209258 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Volume d51a3a0f-f047-4a0e-be70-00f264cc0c50: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d51a3a0f-f047-4a0e-be70-00f264cc0c50', 'volume_size': 1} Sep 27 10:27:09.209808 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] creating volume 'volume-d51a3a0f-f047-4a0e-be70-00f264cc0c50' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:27:09.210667 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:09.364534 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:09.365269 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:09.370528 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (83d0d32f-4743-4d8e-9765-95ecb6418f50) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d51a3a0f-f047-4a0e-be70-00f264cc0c50', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:09.381792 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9a9ed11b-9186-4217-825a-b97531d19024) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:09.405968 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Volume volume-d51a3a0f-f047-4a0e-be70-00f264cc0c50 (d51a3a0f-f047-4a0e-be70-00f264cc0c50): created successfully Sep 27 10:27:09.410898 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9a9ed11b-9186-4217-825a-b97531d19024) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:09.420320 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Flow 'volume_create_manager' (b4ea0d21-db8f-46f8-88da-39bc4b808913) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:09.447169 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ff25861c-c166-40c0-93e6-9fc37bb50a95 tempest-VolumesListAdminTestJSON-396610026 None] Created volume successfully. Sep 27 10:27:10.745409 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:10.770289 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Flow 'volume_create_manager' (fc24e30f-1a8c-4a91-8cd6-a58407109de1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:10.778801 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d308d13a-2f32-4fa2-b3f3-676549840848) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:10.801030 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=62,cinder:UPDATE=29,cinder:INSERT=5,cinder:DELETE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:10.823161 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d308d13a-2f32-4fa2-b3f3-676549840848) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-2135184510',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='99e1389595b74807b615e1c6a289ca10',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:11Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:11Z,use_quota=True,user_id='5ac35bd9df384cfcadd179543142ba80',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:10.830791 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (aa2d4d3a-5da1-4b2e-9877-92fc7330545e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:10.835495 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (aa2d4d3a-5da1-4b2e-9877-92fc7330545e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:10.842838 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6c4ce10a-ba7a-4025-b96d-e7309ee3b837) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:10.847146 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6c4ce10a-ba7a-4025-b96d-e7309ee3b837) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4', 'volume_name': 'volume-4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:10.852643 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e637145c-4ed0-498f-9f1d-8cf9279ba7b4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:10.856753 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e637145c-4ed0-498f-9f1d-8cf9279ba7b4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:10.862394 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f080be32-03b2-45f6-ba55-060cd415e32b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:10.863742 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Volume 4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4', 'volume_size': 1} Sep 27 10:27:10.864378 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] creating volume 'volume-4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:27:10.865386 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:10.964598 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:10.965684 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:10.972716 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f080be32-03b2-45f6-ba55-060cd415e32b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:10.979736 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a9bef560-59ce-434d-8aa9-7bc185fce50c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:10.998444 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Volume volume-4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4 (4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4): created successfully Sep 27 10:27:11.004071 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a9bef560-59ce-434d-8aa9-7bc185fce50c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:11.012250 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Flow 'volume_create_manager' (fc24e30f-1a8c-4a91-8cd6-a58407109de1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:11.024865 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-bb77daf2-e4c5-4baa-9e13-ba4a8682154c tempest-VolumesListAdminTestJSON-396610026 None] Created volume successfully. Sep 27 10:27:11.906216 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=67,cinder:UPDATE=28,cinder:DELETE=2,cinder:INSERT=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:11.931577 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-6bfb6226-7d6a-4222-8133-a490e895d05b req-dbc0ae98-ba5f-4f58-a358-244cd157ade9 tempest-VolumesSnapshotTestJSON-1705236581 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:27:12.767518 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:12.793219 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Flow 'volume_create_manager' (b2e01fdd-1530-4577-a7e1-9b8ce5534b08) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:12.812927 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (664cc65b-d5a8-4b43-b149-6bf4eac19f16) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:12.860576 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (664cc65b-d5a8-4b43-b149-6bf4eac19f16) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:12Z,deleted=False,deleted_at=None,display_description=None,display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f8ab2e22-7bed-4189-8fce-9b12c5df6472,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9c8517e1994049f099d505507358ee6c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:13Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:13Z,use_quota=True,user_id='e78a5831ff154e0d8cd76029b705d941',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:12.883604 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a614ff25-be11-4936-8901-45332528b811) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:12.890879 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a614ff25-be11-4936-8901-45332528b811) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:12.906244 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (88fcd832-81bf-41ac-8f8c-3ee3bc0fbda6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:12.916007 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (88fcd832-81bf-41ac-8f8c-3ee3bc0fbda6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f8ab2e22-7bed-4189-8fce-9b12c5df6472', 'volume_name': 'volume-f8ab2e22-7bed-4189-8fce-9b12c5df6472', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:12.930998 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1355e5e4-2f37-48d8-a428-04957b04c712) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:12.935377 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1355e5e4-2f37-48d8-a428-04957b04c712) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:12.950402 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a504a0bf-1607-4c83-9aa3-83d626edba84) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:12.954865 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Volume f8ab2e22-7bed-4189-8fce-9b12c5df6472: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f8ab2e22-7bed-4189-8fce-9b12c5df6472', 'volume_size': 1} Sep 27 10:27:12.959799 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] creating volume 'volume-f8ab2e22-7bed-4189-8fce-9b12c5df6472' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:27:12.962030 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:13.061188 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-6bfb6226-7d6a-4222-8133-a490e895d05b req-dbc0ae98-ba5f-4f58-a358-244cd157ade9 tempest-VolumesSnapshotTestJSON-1705236581 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.128s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:27:13.062180 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-6bfb6226-7d6a-4222-8133-a490e895d05b req-dbc0ae98-ba5f-4f58-a358-244cd157ade9 tempest-VolumesSnapshotTestJSON-1705236581 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-cd03ad33-8f4c-4084-8b36-72d1fe84c526', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'cd03ad33-8f4c-4084-8b36-72d1fe84c526', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:27:13.099182 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-6bfb6226-7d6a-4222-8133-a490e895d05b req-dbc0ae98-ba5f-4f58-a358-244cd157ade9 tempest-VolumesSnapshotTestJSON-1705236581 None] Connection info returned from driver {'name': 'volumes/volume-cd03ad33-8f4c-4084-8b36-72d1fe84c526', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'cd03ad33-8f4c-4084-8b36-72d1fe84c526', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'e2b72fd4-4c17-427a-a088-db2da7cc3246'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:27:13.138492 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:13.138883 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:13.143085 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a504a0bf-1607-4c83-9aa3-83d626edba84) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f8ab2e22-7bed-4189-8fce-9b12c5df6472', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:13.148107 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cd70606f-0c1b-445b-9646-0e30730646a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:13.167258 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Volume volume-f8ab2e22-7bed-4189-8fce-9b12c5df6472 (f8ab2e22-7bed-4189-8fce-9b12c5df6472): created successfully Sep 27 10:27:13.172557 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cd70606f-0c1b-445b-9646-0e30730646a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:13.180100 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Flow 'volume_create_manager' (b2e01fdd-1530-4577-a7e1-9b8ce5534b08) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:13.195475 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e1cc7e15-70f8-46a1-ad6e-176ce09990d3 tempest-VolumesListAdminTestJSON-1273841817 None] Created volume successfully. Sep 27 10:27:13.220428 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-6bfb6226-7d6a-4222-8133-a490e895d05b req-dbc0ae98-ba5f-4f58-a358-244cd157ade9 tempest-VolumesSnapshotTestJSON-1705236581 None] attachment_update completed successfully. Sep 27 10:27:13.928123 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] Acquiring lock "cinder-f8ab2e22-7bed-4189-8fce-9b12c5df6472-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:13.935999 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] Lock "cinder-f8ab2e22-7bed-4189-8fce-9b12c5df6472-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:13.983062 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:14.054479 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:27:14.055028 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] Volume volume-f8ab2e22-7bed-4189-8fce-9b12c5df6472 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:14.065971 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] deleting rbd volume volume-f8ab2e22-7bed-4189-8fce-9b12c5df6472 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:27:14.100388 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] Acquiring lock "cinder-4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:14.122528 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] Lock "cinder-4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4-delete_volume" acquired by "delete_volume" :: waited 0.021s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:14.221974 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:14.334849 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:27:14.335855 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] Volume volume-4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:14.352328 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] deleting rbd volume volume-4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:27:14.570760 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] Created reservations ['cb0b26db-63f9-4d4b-88ca-3fc4ead69573', '34c9ac54-6dc3-423d-8200-c5c7f84bf52c', 'ae96a551-786c-47ef-91d8-20d12f442d78', 'c613e898-9d52-495f-8295-b7b43ffce5c2'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:14.706326 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:14.816965 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:14.915924 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:27:14.921320 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] Deleted volume successfully. Sep 27 10:27:14.992861 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3ae81626-f5fd-49dd-8c7d-72c87c38949b tempest-VolumesListAdminTestJSON-1273841817 None] Lock "cinder-f8ab2e22-7bed-4189-8fce-9b12c5df6472-delete_volume" released by "delete_volume" :: held 1.055s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:15.013891 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] Created reservations ['d5c40d95-ccc5-42b9-8a36-c5a4497d4f61', '25dc4f04-5dfa-452d-b68d-bb22281083ca', '590f9fba-a90b-4577-bc85-ed4614fcd3f3', 'c2bfa0f4-9463-4bfd-9751-96d62a9cb6f5'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:15.129435 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:15.218498 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:15.303481 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:27:15.321589 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] Deleted volume successfully. Sep 27 10:27:15.357208 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6e3c65c5-db26-4483-a03e-c5dfef67293b tempest-VolumesListAdminTestJSON-396610026 None] Lock "cinder-4bcc5c87-12bf-4917-9bdd-f79c2b32ceb4-delete_volume" released by "delete_volume" :: held 1.236s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:15.407216 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] Acquiring lock "cinder-d51a3a0f-f047-4a0e-be70-00f264cc0c50-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:15.423516 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] Lock "cinder-d51a3a0f-f047-4a0e-be70-00f264cc0c50-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:15.536811 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:15.660832 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:27:15.662648 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] Volume volume-d51a3a0f-f047-4a0e-be70-00f264cc0c50 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:15.676907 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] deleting rbd volume volume-d51a3a0f-f047-4a0e-be70-00f264cc0c50 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:27:15.752741 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-91db0c1f-3c07-4667-81b6-9c45fc285531 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:16.039039 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] Created reservations ['b3ae325b-2ff1-4376-8847-6aae972df804', 'e5286bf9-8645-451a-812b-c722c2765dd2', '479ac2c0-df49-47ca-84e6-7b3712e63045', '720b704d-a85b-4dbe-b677-ce086687b639'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:16.107035 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:16.175817 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:16.242310 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:27:16.259300 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] Deleted volume successfully. Sep 27 10:27:16.284295 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-11195dd5-e5d8-4360-8ae2-244c620b63a6 tempest-VolumesListAdminTestJSON-396610026 None] Lock "cinder-d51a3a0f-f047-4a0e-be70-00f264cc0c50-delete_volume" released by "delete_volume" :: held 0.862s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:16.386885 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-91db0c1f-3c07-4667-81b6-9c45fc285531 tempest-VolumesSnapshotTestJSON-1705236581 None] Create snapshot completed successfully Sep 27 10:27:16.391524 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=60,cinder:DELETE=3,cinder:UPDATE=32,cinder:INSERT=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:16.673790 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] Acquiring lock "cinder-a3cf6aff-9473-4a81-8f1e-1034bda2a859-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:16.684915 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] Lock "cinder-a3cf6aff-9473-4a81-8f1e-1034bda2a859-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:16.731099 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:16.791524 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:27:16.791945 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] Volume volume-a3cf6aff-9473-4a81-8f1e-1034bda2a859 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:16.799478 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] deleting rbd volume volume-a3cf6aff-9473-4a81-8f1e-1034bda2a859 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:27:16.987372 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4bfc55a1-543f-41d1-9917-4b4aa6eb652e tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:17.102621 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] Created reservations ['a818637c-4c28-4593-8865-a4031aa32f43', 'fdb5e003-e5ba-43e6-8421-a5dcfc1c9bbd', 'e2caaba0-db04-47be-aec4-649028ba5727', '5e63250e-64bf-4483-86a0-fe205c6bf785'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:17.112987 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=61,cinder:UPDATE=27,cinder:DELETE=3,cinder:INSERT=9 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:17.153648 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:17.198296 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:17.239756 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:27:17.244629 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] Deleted volume successfully. Sep 27 10:27:17.259211 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d1d6b52b-be41-4a2e-bc9c-0d77907f8988 tempest-VolumesListAdminTestJSON-396610026 None] Lock "cinder-a3cf6aff-9473-4a81-8f1e-1034bda2a859-delete_volume" released by "delete_volume" :: held 0.574s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:17.368635 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-4bfc55a1-543f-41d1-9917-4b4aa6eb652e tempest-VolumesSnapshotTestJSON-1705236581 None] Create snapshot completed successfully Sep 27 10:27:18.108842 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7c8c865f-4075-4e95-83af-48e20995ca69 tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-ba589d23-c36b-4878-b92c-6cafed4e6bf5-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:18.127386 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7c8c865f-4075-4e95-83af-48e20995ca69 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-ba589d23-c36b-4878-b92c-6cafed4e6bf5-delete_snapshot" acquired by "delete_snapshot" :: waited 0.019s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:18.129157 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7c8c865f-4075-4e95-83af-48e20995ca69 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:20.702544 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:20.724971 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Flow 'volume_create_manager' (48ad88e5-e3e1-48ed-b323-2ae7e8255d35) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:20.732003 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (27e326c4-7ec0-4f06-b1c5-7da33d6ed8a5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:20.747926 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (27e326c4-7ec0-4f06-b1c5-7da33d6ed8a5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotMetadataTestJSON-Volume-232812228',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2ccce88d-4804-4ca5-86a0-9b0c3a5507b6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f3b0c51c56bb4f62892ad8ab972fd550',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:21Z,use_quota=True,user_id='920de6c5d8564485968c5bc6a8d742bd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:20.753068 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6173e7a5-9042-4b72-bc32-7702f8b988f4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:20.757717 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6173e7a5-9042-4b72-bc32-7702f8b988f4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:20.764470 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f06681d3-4bf3-4a97-b264-123841b322e4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:20.768824 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f06681d3-4bf3-4a97-b264-123841b322e4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2ccce88d-4804-4ca5-86a0-9b0c3a5507b6', 'volume_name': 'volume-2ccce88d-4804-4ca5-86a0-9b0c3a5507b6', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:20.773639 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (729da561-94b7-47c3-b905-f991c92f8fb6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:20.777386 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (729da561-94b7-47c3-b905-f991c92f8fb6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:20.781891 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f1b17be6-a8b9-4c68-a8da-ce49866e5ea7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:20.782939 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Volume 2ccce88d-4804-4ca5-86a0-9b0c3a5507b6: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2ccce88d-4804-4ca5-86a0-9b0c3a5507b6', 'volume_size': 1} Sep 27 10:27:20.783333 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] creating volume 'volume-2ccce88d-4804-4ca5-86a0-9b0c3a5507b6' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:27:20.784105 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:20.859523 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:20.859935 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:20.863824 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f1b17be6-a8b9-4c68-a8da-ce49866e5ea7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2ccce88d-4804-4ca5-86a0-9b0c3a5507b6', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:20.868448 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d410a3b5-b32a-4e60-a0fc-972963c876b6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:20.879857 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Volume volume-2ccce88d-4804-4ca5-86a0-9b0c3a5507b6 (2ccce88d-4804-4ca5-86a0-9b0c3a5507b6): created successfully Sep 27 10:27:20.885008 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d410a3b5-b32a-4e60-a0fc-972963c876b6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:20.890484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Flow 'volume_create_manager' (48ad88e5-e3e1-48ed-b323-2ae7e8255d35) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:20.909998 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-2c90b4fc-76a0-42b6-9583-7ae08ab1f8d3 tempest-SnapshotMetadataTestJSON-73856879 None] Created volume successfully. Sep 27 10:27:22.028917 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-eeec2d06-b6f3-4d8c-8d3a-141718e195d7 tempest-SnapshotMetadataTestJSON-73856879 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:22.476804 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-eeec2d06-b6f3-4d8c-8d3a-141718e195d7 tempest-SnapshotMetadataTestJSON-73856879 None] Create snapshot completed successfully Sep 27 10:27:24.080536 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0aad4c97-0675-4753-a6d8-418a507cdb3f tempest-SnapshotMetadataTestJSON-73856879 None] Acquiring lock "cinder-c97c7fc4-c58e-44ba-89ee-db2bc51e2be6-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:24.095084 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0aad4c97-0675-4753-a6d8-418a507cdb3f tempest-SnapshotMetadataTestJSON-73856879 None] Lock "cinder-c97c7fc4-c58e-44ba-89ee-db2bc51e2be6-delete_snapshot" acquired by "delete_snapshot" :: waited 0.014s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:24.097131 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0aad4c97-0675-4753-a6d8-418a507cdb3f tempest-SnapshotMetadataTestJSON-73856879 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:24.529608 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-0aad4c97-0675-4753-a6d8-418a507cdb3f tempest-SnapshotMetadataTestJSON-73856879 None] Created reservations ['ef234cba-33af-4655-a31b-81412ebcc761', '96aa532c-8f45-49ae-aeb2-943b724f691a', 'b0b56d3d-b321-427b-8c2e-3d52d5b23824', '38d644c8-80d3-481f-b7a0-56a86fc9e712'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:24.579588 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0aad4c97-0675-4753-a6d8-418a507cdb3f tempest-SnapshotMetadataTestJSON-73856879 None] Delete snapshot completed successfully. Sep 27 10:27:24.589764 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0aad4c97-0675-4753-a6d8-418a507cdb3f tempest-SnapshotMetadataTestJSON-73856879 None] Lock "cinder-c97c7fc4-c58e-44ba-89ee-db2bc51e2be6-delete_snapshot" released by "delete_snapshot" :: held 0.495s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:25.273401 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] Acquiring lock "cinder-2ccce88d-4804-4ca5-86a0-9b0c3a5507b6-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:25.314674 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] Lock "cinder-2ccce88d-4804-4ca5-86a0-9b0c3a5507b6-delete_volume" acquired by "delete_volume" :: waited 0.041s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:25.379830 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:25.452587 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:27:25.453242 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] Volume volume-2ccce88d-4804-4ca5-86a0-9b0c3a5507b6 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:25.463491 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] deleting rbd volume volume-2ccce88d-4804-4ca5-86a0-9b0c3a5507b6 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:27:25.828023 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] Created reservations ['68ded44d-97e2-4b5a-88ca-aa85bdbbc364', '2469ee39-4922-4651-9be3-6759eec03038', 'f9945d09-9404-4102-bbac-1dfac911bba2', 'fd16a7bb-2b4e-4cd5-ba8f-a84a966d1ea9'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:25.898443 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:25.961911 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:26.017380 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:27:26.033266 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] Deleted volume successfully. Sep 27 10:27:26.055642 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-72dc9b06-d6f3-491e-a45a-7a076f4a6b72 tempest-SnapshotMetadataTestJSON-73856879 None] Lock "cinder-2ccce88d-4804-4ca5-86a0-9b0c3a5507b6-delete_volume" released by "delete_volume" :: held 0.741s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:29.306487 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:29.328188 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Flow 'volume_create_manager' (c36295e2-5f89-4b12-af4e-d3f7cf1588e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:29.334887 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2fb5d362-e0be-41c3-8c67-5df0bb3bfbb9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:29.353274 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2fb5d362-e0be-41c3-8c67-5df0bb3bfbb9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-2098278197',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b4bdde26-62c0-4855-b2d5-109b9bdebba7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fade501d78244c13bd31fddc884c8981',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:29Z,use_quota=True,user_id='0b207e8521b74e7bb783b9dfbc8f98ad',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:29.358541 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f0d11cdc-0b3d-4cde-bc54-8754470e2f47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:29.362660 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f0d11cdc-0b3d-4cde-bc54-8754470e2f47) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:29.368136 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b57d5e8a-a398-4132-843b-c7c9172626b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:29.372281 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b57d5e8a-a398-4132-843b-c7c9172626b1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b4bdde26-62c0-4855-b2d5-109b9bdebba7', 'volume_name': 'volume-b4bdde26-62c0-4855-b2d5-109b9bdebba7', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:29.377397 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c8b32ae2-b39d-4cfa-8ea5-aa87ca0ddd47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:29.381602 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c8b32ae2-b39d-4cfa-8ea5-aa87ca0ddd47) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:29.386592 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca92695b-0d16-45b0-9f64-2b6d4e536962) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:29.387690 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Volume b4bdde26-62c0-4855-b2d5-109b9bdebba7: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b4bdde26-62c0-4855-b2d5-109b9bdebba7', 'volume_size': 1} Sep 27 10:27:29.388085 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] creating volume 'volume-b4bdde26-62c0-4855-b2d5-109b9bdebba7' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:27:29.389135 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:29.455817 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:29.456472 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:29.460752 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca92695b-0d16-45b0-9f64-2b6d4e536962) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b4bdde26-62c0-4855-b2d5-109b9bdebba7', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:29.467770 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4b1fd403-60a1-4561-b14f-7559ba6c9533) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:29.481391 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Volume volume-b4bdde26-62c0-4855-b2d5-109b9bdebba7 (b4bdde26-62c0-4855-b2d5-109b9bdebba7): created successfully Sep 27 10:27:29.486949 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4b1fd403-60a1-4561-b14f-7559ba6c9533) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:29.497561 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Flow 'volume_create_manager' (c36295e2-5f89-4b12-af4e-d3f7cf1588e5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:29.513086 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-33fceeea-33e8-4136-8272-2c68aa530e3f tempest-VolumesMetadataTest-353802058 None] Created volume successfully. Sep 27 10:27:29.790270 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:27:29.791973 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:29.853667 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:29.921524 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:27:31.082708 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] Acquiring lock "cinder-b4bdde26-62c0-4855-b2d5-109b9bdebba7-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:31.101967 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] Lock "cinder-b4bdde26-62c0-4855-b2d5-109b9bdebba7-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:31.106615 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=60,cinder:UPDATE=30,cinder:INSERT=9,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:31.172776 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:31.223967 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:27:31.224945 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] Volume volume-b4bdde26-62c0-4855-b2d5-109b9bdebba7 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:31.233458 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] deleting rbd volume volume-b4bdde26-62c0-4855-b2d5-109b9bdebba7 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:27:31.503969 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] Created reservations ['3c7ff215-2b24-4270-b479-b0f8e2388199', 'b00855e6-acf7-4a01-b34e-22d2c716613d', 'ffa9f4ef-a134-49d3-a999-3556bc0336ba', 'ae72d46c-1d30-44b1-b572-90873eab5257'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:31.568828 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:31.610183 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:31.672802 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:27:31.678170 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] Deleted volume successfully. Sep 27 10:27:31.695321 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d4af5667-e395-48d2-b364-c82971845929 tempest-VolumesMetadataTest-353802058 None] Lock "cinder-b4bdde26-62c0-4855-b2d5-109b9bdebba7-delete_volume" released by "delete_volume" :: held 0.593s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:35.700482 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:35.731953 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Flow 'volume_create_manager' (c14cb656-72cd-46d8-b85b-9d0ce2d2dea0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:35.749485 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3f114765-7e9d-4cc1-b1c9-bc12fa54ade6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:35.768566 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3f114765-7e9d-4cc1-b1c9-bc12fa54ade6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:35Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersTest-Volume-1572292077',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=15068e60-586d-4ae5-9a54-bf7f9372c75a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='7735186883cb4234bbb276780101cecb',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:36Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:36Z,use_quota=True,user_id='e2a8653304ce4425971752f838a0ea01',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:35.780007 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (69ab127b-5e95-4253-bd83-3780211fb9c2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:35.784210 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (69ab127b-5e95-4253-bd83-3780211fb9c2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:35.789487 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ba24d802-5d86-4c62-b182-1a6c7bb77dad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:35.794506 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ba24d802-5d86-4c62-b182-1a6c7bb77dad) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '15068e60-586d-4ae5-9a54-bf7f9372c75a', 'volume_name': 'volume-15068e60-586d-4ae5-9a54-bf7f9372c75a', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:35.804229 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0a363212-ee60-4ab6-a22e-2b611c13aceb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:35.810532 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0a363212-ee60-4ab6-a22e-2b611c13aceb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:35.818467 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2112b153-e69c-4ab4-a7ec-dfbbc77eab79) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:35.819926 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Volume 15068e60-586d-4ae5-9a54-bf7f9372c75a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-15068e60-586d-4ae5-9a54-bf7f9372c75a', 'volume_size': 1} Sep 27 10:27:35.820487 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] creating volume 'volume-15068e60-586d-4ae5-9a54-bf7f9372c75a' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:27:35.821629 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:35.889546 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:35.889921 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:35.893824 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2112b153-e69c-4ab4-a7ec-dfbbc77eab79) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-15068e60-586d-4ae5-9a54-bf7f9372c75a', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:35.898842 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6b9fe1a2-280b-42ff-bced-63e56e1cdb58) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:35.909582 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Volume volume-15068e60-586d-4ae5-9a54-bf7f9372c75a (15068e60-586d-4ae5-9a54-bf7f9372c75a): created successfully Sep 27 10:27:35.913490 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6b9fe1a2-280b-42ff-bced-63e56e1cdb58) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:35.919327 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Flow 'volume_create_manager' (c14cb656-72cd-46d8-b85b-9d0ce2d2dea0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:35.930541 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ae9dddac-9c4e-4c0f-bf39-3ed7e799d2d7 tempest-VolumesTransfersTest-1461924559 None] Created volume successfully. Sep 27 10:27:37.328776 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d2daa989-c47f-4bf8-a074-86a4d2672d49 tempest-VolumesTransfersTest-872762508 None] Transfer volume completed successfully. Sep 27 10:27:37.973307 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] Acquiring lock "cinder-15068e60-586d-4ae5-9a54-bf7f9372c75a-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:37.985300 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] Lock "cinder-15068e60-586d-4ae5-9a54-bf7f9372c75a-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:38.020441 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:38.062491 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:27:38.063044 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] Volume volume-15068e60-586d-4ae5-9a54-bf7f9372c75a is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:38.069576 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] deleting rbd volume volume-15068e60-586d-4ae5-9a54-bf7f9372c75a {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:27:38.347345 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=59,cinder:UPDATE=34,cinder:DELETE=5,cinder:INSERT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:38.377144 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] Created reservations ['b30157ff-4504-4809-9d1d-a7266fd253f5', 'f18304ff-356f-424b-b4d5-16d2f13f7126', '6a2ab9e0-1f5f-4650-af78-b552034f06cf', '1fc37b57-ba82-45b9-8add-b1ce5308b610'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:38.434132 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:38.469532 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:38.500170 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:27:38.505724 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] Deleted volume successfully. Sep 27 10:27:38.523917 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-dd554313-0502-4bd8-87fe-ece64b512880 tempest-VolumesTransfersTest-1824030447 None] Lock "cinder-15068e60-586d-4ae5-9a54-bf7f9372c75a-delete_volume" released by "delete_volume" :: held 0.539s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:39.509209 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:39.531511 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Flow 'volume_create_manager' (f712d82e-b795-4f98-bf69-1f93d898587e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:39.538600 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (79da8014-d2cc-4b99-87c7-b11ad6ff6f2c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:39.560220 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (79da8014-d2cc-4b99-87c7-b11ad6ff6f2c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:39Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersTest-Volume-6179531',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1ee51bf8-9b68-4385-81e9-f2181196bb8c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='7735186883cb4234bbb276780101cecb',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:39Z,use_quota=True,user_id='e2a8653304ce4425971752f838a0ea01',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:39.566392 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c5d09009-632b-4d6b-9fd5-2e12d06f0934) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:39.570838 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c5d09009-632b-4d6b-9fd5-2e12d06f0934) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:39.575971 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (89c68db9-1d60-4126-94e3-0b37ae930867) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:39.587045 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (89c68db9-1d60-4126-94e3-0b37ae930867) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1ee51bf8-9b68-4385-81e9-f2181196bb8c', 'volume_name': 'volume-1ee51bf8-9b68-4385-81e9-f2181196bb8c', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:39.610819 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b0f012bc-6523-4300-b369-994d521866f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:39.616916 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b0f012bc-6523-4300-b369-994d521866f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:39.622109 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6184443b-47ef-46ad-8b7d-c6cf0c31095b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:39.623797 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Volume 1ee51bf8-9b68-4385-81e9-f2181196bb8c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1ee51bf8-9b68-4385-81e9-f2181196bb8c', 'volume_size': 1} Sep 27 10:27:39.624398 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] creating volume 'volume-1ee51bf8-9b68-4385-81e9-f2181196bb8c' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:27:39.625695 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:39.748574 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:39.749388 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:39.755271 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6184443b-47ef-46ad-8b7d-c6cf0c31095b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1ee51bf8-9b68-4385-81e9-f2181196bb8c', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:39.765873 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (419ad780-f9f1-4143-a915-fcd8747cab99) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:39.783567 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Volume volume-1ee51bf8-9b68-4385-81e9-f2181196bb8c (1ee51bf8-9b68-4385-81e9-f2181196bb8c): created successfully Sep 27 10:27:39.791348 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (419ad780-f9f1-4143-a915-fcd8747cab99) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:39.800501 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Flow 'volume_create_manager' (f712d82e-b795-4f98-bf69-1f93d898587e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:39.813941 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-81908c40-a9f5-4a63-9c66-0977a8fb9d9e tempest-VolumesTransfersTest-1461924559 None] Created volume successfully. Sep 27 10:27:41.079539 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] Acquiring lock "cinder-1ee51bf8-9b68-4385-81e9-f2181196bb8c-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:41.095282 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] Lock "cinder-1ee51bf8-9b68-4385-81e9-f2181196bb8c-delete_volume" acquired by "delete_volume" :: waited 0.017s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:41.130954 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:41.193559 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:27:41.194257 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] Volume volume-1ee51bf8-9b68-4385-81e9-f2181196bb8c is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:41.200497 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] deleting rbd volume volume-1ee51bf8-9b68-4385-81e9-f2181196bb8c {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:27:41.592080 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] Created reservations ['85e37bd6-f654-4fb1-b820-86758df05c51', 'eedb4dff-caaf-4cd0-8a21-5f1d637c4266', '5a712b91-4003-4fc5-8b7b-052b8df48762', '08961d76-d50d-4562-87c2-d1b9aac78204'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:41.669584 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:41.717105 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:41.758696 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:27:41.765350 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] Deleted volume successfully. Sep 27 10:27:41.783671 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b9b39e0f-643a-4665-ac78-5c3f7ca32421 tempest-VolumesTransfersTest-1824030447 None] Lock "cinder-1ee51bf8-9b68-4385-81e9-f2181196bb8c-delete_volume" released by "delete_volume" :: held 0.688s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:47.228415 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:47.260531 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Flow 'volume_create_manager' (8ba5b6dc-36f0-4b65-82cc-5a3ad4fe5cef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:47.268958 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (42d9a774-6057-4a33-950d-9144fa6c45d6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:47.287918 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (42d9a774-6057-4a33-950d-9144fa6c45d6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-2042058998',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=30c5d9d5-6cbf-437f-998a-6189a5200e4c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b1b9805b22374f82b3e46883f786d963',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:47Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:47Z,use_quota=True,user_id='da3aef8b80bb411eb325010a4744f350',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:47.293799 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6503102c-87bd-48a4-940c-c9a2237c5c7d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:47.298534 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6503102c-87bd-48a4-940c-c9a2237c5c7d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:47.305008 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (52fa5456-3af6-4962-8605-55ba7629b0c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:47.511751 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (52fa5456-3af6-4962-8605-55ba7629b0c3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '30c5d9d5-6cbf-437f-998a-6189a5200e4c', 'volume_name': 'volume-30c5d9d5-6cbf-437f-998a-6189a5200e4c', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:47.524350 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fa63fb8b-c74c-42ae-810f-b645d431d709) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:47.525197 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fa63fb8b-c74c-42ae-810f-b645d431d709) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:47.532332 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bcee9745-55fc-4a9f-a36a-1c3fa0ecf36a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:47.534020 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Volume 30c5d9d5-6cbf-437f-998a-6189a5200e4c: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-30c5d9d5-6cbf-437f-998a-6189a5200e4c', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:27:47.534882 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Cloning 30c5d9d5-6cbf-437f-998a-6189a5200e4c from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:27:47.535977 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:47.552366 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.015s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:47.565584 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 27, 47, 557331)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:27:47.566266 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:27:47.566730 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:27:47.577117 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.025s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:47.578951 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:27:47.594036 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 27, 47, 589144)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:27:47.594671 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:27:47.595110 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:27:47.611774 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:47.696215 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] creating snapshot='volume-30c5d9d5-6cbf-437f-998a-6189a5200e4c.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:27:48.343033 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-30c5d9d5-6cbf-437f-998a-6189a5200e4c.clone_snap' to 'volume-30c5d9d5-6cbf-437f-998a-6189a5200e4c' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:27:48.406879 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=59,cinder:INSERT=9,cinder:UPDATE=28,cinder:DELETE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:48.482222 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-7c8c865f-4075-4e95-83af-48e20995ca69 tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['8cdc2a23-641d-45e0-b691-b98d3d99b236', '825080d5-f892-4b22-a883-d74e95ec7614', '0f6a858f-4390-44bb-a8e0-d27344a23ce9', '6f5cfbe3-7837-4dd1-abc7-84fbb143fd3a'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:48.557423 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7c8c865f-4075-4e95-83af-48e20995ca69 tempest-VolumesSnapshotTestJSON-1705236581 None] Delete snapshot completed successfully. Sep 27 10:27:48.564443 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7c8c865f-4075-4e95-83af-48e20995ca69 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-ba589d23-c36b-4878-b92c-6cafed4e6bf5-delete_snapshot" released by "delete_snapshot" :: held 30.437s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:48.649370 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:48.651226 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:48.651598 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:48.716783 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:27:48.716783 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Marking volume 30c5d9d5-6cbf-437f-998a-6189a5200e4c as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:27:48.731363 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 30c5d9d5-6cbf-437f-998a-6189a5200e4c. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:27:48.731703 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Creating volume glance metadata for volume 30c5d9d5-6cbf-437f-998a-6189a5200e4c backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:27:48.837492 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bcee9745-55fc-4a9f-a36a-1c3fa0ecf36a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-30c5d9d5-6cbf-437f-998a-6189a5200e4c', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:48.851114 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ef72c0a3-f67e-4323-b173-323df18e582f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:48.875399 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Volume volume-30c5d9d5-6cbf-437f-998a-6189a5200e4c (30c5d9d5-6cbf-437f-998a-6189a5200e4c): created successfully Sep 27 10:27:48.881600 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ef72c0a3-f67e-4323-b173-323df18e582f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:48.888084 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Flow 'volume_create_manager' (8ba5b6dc-36f0-4b65-82cc-5a3ad4fe5cef) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:48.902977 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d2d3ba6e-e753-444e-a621-dee4871083db tempest-VolumesCloneTest-1545028351 None] Created volume successfully. Sep 27 10:27:49.605554 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b0d04910-2466-4e1f-9aa4-4be78a1f3e4b tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-bb7f77a5-6b9d-4a5f-91ee-0ea2a95c50ae-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:49.625151 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b0d04910-2466-4e1f-9aa4-4be78a1f3e4b tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-bb7f77a5-6b9d-4a5f-91ee-0ea2a95c50ae-delete_snapshot" acquired by "delete_snapshot" :: waited 0.026s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:49.633098 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0d04910-2466-4e1f-9aa4-4be78a1f3e4b tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:50.118579 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:50.157833 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Flow 'volume_create_manager' (8cdc3ab9-80ba-457a-b68c-9c92a30c53af) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:50.168402 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (08ebf928-f39f-438e-9898-2c0bded65b8f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:50.188244 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (08ebf928-f39f-438e-9898-2c0bded65b8f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-179110381',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=9c1dd36e-0272-4926-8d1e-4a09b8d41b05,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b1b9805b22374f82b3e46883f786d963',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:50Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=30c5d9d5-6cbf-437f-998a-6189a5200e4c,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:50Z,use_quota=True,user_id='da3aef8b80bb411eb325010a4744f350',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:50.193837 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f4cda642-b9ca-4382-a931-55f6705499eb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:50.201243 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f4cda642-b9ca-4382-a931-55f6705499eb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:50.206637 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (807a2079-c2e7-4f42-9bad-a5378f95349c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:50.228260 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (807a2079-c2e7-4f42-9bad-a5378f95349c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '9c1dd36e-0272-4926-8d1e-4a09b8d41b05', 'volume_name': 'volume-9c1dd36e-0272-4926-8d1e-4a09b8d41b05', 'volume_size': 1, 'source_volid': '30c5d9d5-6cbf-437f-998a-6189a5200e4c', 'source_volstatus': 'available'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:50.233844 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (db82f64b-1e38-4dde-b4a5-410618830eff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:50.240753 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (db82f64b-1e38-4dde-b4a5-410618830eff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:50.245814 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8311e39b-71c1-4b2f-b862-9a53f862e501) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:50.247043 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Volume 9c1dd36e-0272-4926-8d1e-4a09b8d41b05: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-9c1dd36e-0272-4926-8d1e-4a09b8d41b05', 'volume_size': 1, 'source_volid': '30c5d9d5-6cbf-437f-998a-6189a5200e4c', 'source_volstatus': 'available'} Sep 27 10:27:50.265190 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:50.397292 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] creating snapshot='volume-9c1dd36e-0272-4926-8d1e-4a09b8d41b05.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:27:51.468255 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] cloning 'volume-30c5d9d5-6cbf-437f-998a-6189a5200e4c@volume-9c1dd36e-0272-4926-8d1e-4a09b8d41b05.clone_snap' to 'volume-9c1dd36e-0272-4926-8d1e-4a09b8d41b05' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:27:51.857431 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:51.858743 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:51.859121 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:51.890847 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:27:51.891597 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Marking volume 9c1dd36e-0272-4926-8d1e-4a09b8d41b05 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:27:51.910363 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Copying metadata from source volume 30c5d9d5-6cbf-437f-998a-6189a5200e4c to 9c1dd36e-0272-4926-8d1e-4a09b8d41b05. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:440}} Sep 27 10:27:51.967434 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8311e39b-71c1-4b2f-b862-9a53f862e501) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9c1dd36e-0272-4926-8d1e-4a09b8d41b05', 'volume_size': 1, 'source_volid': '30c5d9d5-6cbf-437f-998a-6189a5200e4c', 'source_volstatus': 'available'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:51.973360 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f7d08972-c637-4a15-b0ff-363a292d7e4d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:51.993341 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Volume volume-9c1dd36e-0272-4926-8d1e-4a09b8d41b05 (9c1dd36e-0272-4926-8d1e-4a09b8d41b05): created successfully Sep 27 10:27:52.000395 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f7d08972-c637-4a15-b0ff-363a292d7e4d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:52.009092 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Flow 'volume_create_manager' (8cdc3ab9-80ba-457a-b68c-9c92a30c53af) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:52.033025 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d00b9e5d-060b-4144-9737-17148c2cb56b tempest-VolumesCloneTest-1545028351 None] Created volume successfully. Sep 27 10:27:52.935292 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:52.956097 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Flow 'volume_create_manager' (8f9eff14-127b-4ac3-b486-53f504c25d45) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:52.978612 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e0de95ef-df13-4c2b-8143-fa49b86dc769) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:52.999510 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e0de95ef-df13-4c2b-8143-fa49b86dc769) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:53Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1951742214',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a58acae5-838b-49f3-8649-f9d383b22759,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b1b9805b22374f82b3e46883f786d963',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:53Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:53Z,use_quota=True,user_id='da3aef8b80bb411eb325010a4744f350',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:53.004985 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ce9b265e-5227-4f13-bbeb-613c74148473) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:53.016617 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ce9b265e-5227-4f13-bbeb-613c74148473) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:53.024982 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff94152c-b134-4200-bb22-40c86b92e88e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:53.029527 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff94152c-b134-4200-bb22-40c86b92e88e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a58acae5-838b-49f3-8649-f9d383b22759', 'volume_name': 'volume-a58acae5-838b-49f3-8649-f9d383b22759', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:53.034743 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0f5e859e-9eea-41cf-83ed-8219f42fd3fe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:53.040108 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0f5e859e-9eea-41cf-83ed-8219f42fd3fe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:53.045296 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c5696787-b275-42f9-bd2c-b9664f6728fd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:53.046556 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Volume a58acae5-838b-49f3-8649-f9d383b22759: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a58acae5-838b-49f3-8649-f9d383b22759', 'volume_size': 1} Sep 27 10:27:53.047106 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] creating volume 'volume-a58acae5-838b-49f3-8649-f9d383b22759' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:27:53.048027 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:53.163916 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:53.164760 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:53.168507 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c5696787-b275-42f9-bd2c-b9664f6728fd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a58acae5-838b-49f3-8649-f9d383b22759', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:53.173214 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1d467bbe-e1ba-47dd-8bf4-0279aabb2c39) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:53.178293 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=27,cinder:SELECT=57,cinder:DELETE=1,cinder:INSERT=15 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:53.188245 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Volume volume-a58acae5-838b-49f3-8649-f9d383b22759 (a58acae5-838b-49f3-8649-f9d383b22759): created successfully Sep 27 10:27:53.192351 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1d467bbe-e1ba-47dd-8bf4-0279aabb2c39) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:53.199298 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Flow 'volume_create_manager' (8f9eff14-127b-4ac3-b486-53f504c25d45) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:53.211553 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ac6200a2-f15a-4090-b287-bbaf979b30da tempest-VolumesCloneTest-1545028351 None] Created volume successfully. Sep 27 10:27:54.233452 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:27:54.293792 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Flow 'volume_create_manager' (7d44ce96-c0e0-4f54-b1b7-d806ad5043eb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:54.300322 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (43688028-9af9-4ee6-8e57-edf470040c40) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:54.316986 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (43688028-9af9-4ee6-8e57-edf470040c40) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:27:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1145060118',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ea655048-895f-4802-ba63-589dfaa8626c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b1b9805b22374f82b3e46883f786d963',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:27:54Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=a58acae5-838b-49f3-8649-f9d383b22759,status='creating',terminated_at=None,updated_at=2022-09-27T10:27:54Z,use_quota=True,user_id='da3aef8b80bb411eb325010a4744f350',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:54.323569 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (576cf080-8275-425e-99f9-266d6347b7db) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:54.329372 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (576cf080-8275-425e-99f9-266d6347b7db) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:54.336556 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (08cd1bb4-ae14-4e46-a477-717286c362e6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:54.360797 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (08cd1bb4-ae14-4e46-a477-717286c362e6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': 'ea655048-895f-4802-ba63-589dfaa8626c', 'volume_name': 'volume-ea655048-895f-4802-ba63-589dfaa8626c', 'volume_size': 2, 'source_volid': 'a58acae5-838b-49f3-8649-f9d383b22759', 'source_volstatus': 'available'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:54.366270 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a71f2ade-b896-4037-a8e6-24e8b8c02ad8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:54.370442 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a71f2ade-b896-4037-a8e6-24e8b8c02ad8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:54.375354 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ff18bc54-09c8-44cd-84a1-1ab1a9704928) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:54.377272 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Volume ea655048-895f-4802-ba63-589dfaa8626c: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-ea655048-895f-4802-ba63-589dfaa8626c', 'volume_size': 2, 'source_volid': 'a58acae5-838b-49f3-8649-f9d383b22759', 'source_volstatus': 'available'} Sep 27 10:27:54.409729 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:54.469784 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] creating snapshot='volume-ea655048-895f-4802-ba63-589dfaa8626c.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:27:55.495438 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] cloning 'volume-a58acae5-838b-49f3-8649-f9d383b22759@volume-ea655048-895f-4802-ba63-589dfaa8626c.clone_snap' to 'volume-ea655048-895f-4802-ba63-589dfaa8626c' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:27:55.663107 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Volume volume-a58acae5-838b-49f3-8649-f9d383b22759 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:55.664495 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:27:55.665089 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:27:55.696183 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] resize volume 'volume-ea655048-895f-4802-ba63-589dfaa8626c' from 1 to 2 {{(pid=138473) _extend_if_required /opt/stack/cinder/cinder/volume/drivers/rbd.py:812}} Sep 27 10:27:55.697224 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:55.835876 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:27:55.840239 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ff18bc54-09c8-44cd-84a1-1ab1a9704928) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ea655048-895f-4802-ba63-589dfaa8626c', 'volume_size': 2, 'source_volid': 'a58acae5-838b-49f3-8649-f9d383b22759', 'source_volstatus': 'available'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:55.845069 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ce662812-8567-4104-b0cc-1d1d5124bf23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:27:55.857652 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Volume volume-ea655048-895f-4802-ba63-589dfaa8626c (ea655048-895f-4802-ba63-589dfaa8626c): created successfully Sep 27 10:27:55.862283 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ce662812-8567-4104-b0cc-1d1d5124bf23) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:27:55.868190 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Flow 'volume_create_manager' (7d44ce96-c0e0-4f54-b1b7-d806ad5043eb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:27:55.881914 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-a4b173fe-62fd-4623-9d1d-415ce68c5bf3 tempest-VolumesCloneTest-1545028351 None] Created volume successfully. Sep 27 10:27:56.424943 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] Acquiring lock "cinder-ea655048-895f-4802-ba63-589dfaa8626c-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:56.435051 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] Lock "cinder-ea655048-895f-4802-ba63-589dfaa8626c-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:56.476101 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:56.533361 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:27:56.543234 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] deleting rbd volume volume-ea655048-895f-4802-ba63-589dfaa8626c {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:27:56.755597 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:27:56.780486 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] Volume volume-a58acae5-838b-49f3-8649-f9d383b22759 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:56.781408 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] deleting parent snapshot volume-ea655048-895f-4802-ba63-589dfaa8626c.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:27:57.644283 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] Created reservations ['fe35f23b-52a8-413f-86e2-5143abffa2f6', '6e38e49f-c1cc-4cf5-b812-eb9f1f84250d', 'e19e8bf3-cd40-4d8f-b4fd-9ab83b6dcb9f', 'cad22098-4917-43a3-bcfd-66258e3609df'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:57.710102 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:57.749933 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:57.783820 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:27:57.789516 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] Deleted volume successfully. Sep 27 10:27:57.801986 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a84aaa62-de30-4c8d-9215-f640321ec4d1 tempest-VolumesCloneTest-1545028351 None] Lock "cinder-ea655048-895f-4802-ba63-589dfaa8626c-delete_volume" released by "delete_volume" :: held 1.367s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:58.667819 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] Acquiring lock "cinder-a58acae5-838b-49f3-8649-f9d383b22759-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:58.680813 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] Lock "cinder-a58acae5-838b-49f3-8649-f9d383b22759-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:58.719055 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:58.758343 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:27:58.758665 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] Volume volume-a58acae5-838b-49f3-8649-f9d383b22759 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:27:58.764327 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] deleting rbd volume volume-a58acae5-838b-49f3-8649-f9d383b22759 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:27:58.977498 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] Created reservations ['433a359d-2664-467e-aaf7-43406936b3ac', 'ecbbb7a7-5fb9-4b04-83fe-0d98244d9185', '098c11b4-0bad-4169-b75a-df8ad648b96e', '24c606fa-a65f-42c9-ae4e-9e29ba9bc075'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:27:58.992496 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=55,cinder:INSERT=19,cinder:UPDATE=21,cinder:DELETE=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:59.029985 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:59.065484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:59.100551 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:27:59.105033 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] Deleted volume successfully. Sep 27 10:27:59.119798 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-4f2c6ff1-7771-42aa-814e-568ae4447356 tempest-VolumesCloneTest-1545028351 None] Lock "cinder-a58acae5-838b-49f3-8649-f9d383b22759-delete_volume" released by "delete_volume" :: held 0.439s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:27:59.822539 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] Acquiring lock "cinder-9c1dd36e-0272-4926-8d1e-4a09b8d41b05-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:27:59.834998 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] Lock "cinder-9c1dd36e-0272-4926-8d1e-4a09b8d41b05-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:27:59.868876 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:27:59.954095 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:27:59.962397 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] deleting rbd volume volume-9c1dd36e-0272-4926-8d1e-4a09b8d41b05 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:28:00.137486 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:28:00.183978 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] deleting parent snapshot volume-9c1dd36e-0272-4926-8d1e-4a09b8d41b05.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:28:00.631960 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] Created reservations ['19edfa3d-95b7-4dc9-b2d4-1c23a8b0de8b', '9d3e74a1-b7dc-401d-ba55-11f08ddabaf7', '20354926-cfaa-4208-af94-68ab9c09c261', 'b4486e53-a189-4582-bb8e-fdb26c8f10e2'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:28:00.686400 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:28:00.709669 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:28:00.734932 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:28:00.738849 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] Deleted volume successfully. Sep 27 10:28:00.754376 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1c979691-1436-4509-8f17-6387e1a535db tempest-VolumesCloneTest-1545028351 None] Lock "cinder-9c1dd36e-0272-4926-8d1e-4a09b8d41b05-delete_volume" released by "delete_volume" :: held 0.919s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:28:00.974397 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] Acquiring lock "cinder-30c5d9d5-6cbf-437f-998a-6189a5200e4c-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:28:00.983833 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] Lock "cinder-30c5d9d5-6cbf-437f-998a-6189a5200e4c-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:28:01.018038 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:28:01.078404 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:28:01.086704 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] deleting rbd volume volume-30c5d9d5-6cbf-437f-998a-6189a5200e4c {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:28:01.309913 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:28:01.341198 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] deleting parent snapshot volume-30c5d9d5-6cbf-437f-998a-6189a5200e4c.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:28:01.660980 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] Created reservations ['91727015-198b-4439-a8c1-5ee0371ef9b4', 'cb03bb11-f32f-4772-9357-02be545f799e', '8a613c80-646a-47ce-9b5d-30b23ba94f0a', 'b3fcae39-6771-437f-80d6-3ab3aaba0407'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:28:01.713704 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=38,cinder:SELECT=53,cinder:INSERT=8,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:01.724720 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:28:01.750965 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:28:01.779116 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:28:01.783872 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] Deleted volume successfully. Sep 27 10:28:01.799570 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-bd6ff374-c763-400f-abdf-55c1a08acac5 tempest-VolumesCloneTest-1545028351 None] Lock "cinder-30c5d9d5-6cbf-437f-998a-6189a5200e4c-delete_volume" released by "delete_volume" :: held 0.816s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:28:17.574746 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:19.843461 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-b0d04910-2466-4e1f-9aa4-4be78a1f3e4b tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['60b9c373-4c51-4c84-a8ef-e46c39f06165', '99f1f345-d27e-421f-9b13-17b7f76d1333', 'ec4a7cd0-5d3d-40fe-9918-b6fa44847a4a', 'dce8c52c-2480-4bb7-8fef-75d95df60ba1'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:28:19.883235 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b0d04910-2466-4e1f-9aa4-4be78a1f3e4b tempest-VolumesSnapshotTestJSON-1705236581 None] Delete snapshot completed successfully. Sep 27 10:28:19.888130 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b0d04910-2466-4e1f-9aa4-4be78a1f3e4b tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-bb7f77a5-6b9d-4a5f-91ee-0ea2a95c50ae-delete_snapshot" released by "delete_snapshot" :: held 30.263s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:28:20.855846 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6475cd4b-3005-4bba-98df-9adf739871ef tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-de9e38c1-d557-40ab-9b62-975274a202db-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:28:20.868412 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6475cd4b-3005-4bba-98df-9adf739871ef tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-de9e38c1-d557-40ab-9b62-975274a202db-delete_snapshot" acquired by "delete_snapshot" :: waited 0.013s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:28:20.870697 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6475cd4b-3005-4bba-98df-9adf739871ef tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:28:29.793463 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:28:29.794266 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:28:29.827574 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:28:29.893823 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:28:37.583400 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=30,cinder:DELETE=2,cinder:SELECT=42,cinder:INSERT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:47.453312 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:28:47.473153 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Flow 'volume_create_manager' (1123d9e4-0421-4b79-afc0-28a8aeab1ea8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:28:47.479175 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cf5a75f9-a97b-4485-bac7-41e7a5443eea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:47.496452 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cf5a75f9-a97b-4485-bac7-41e7a5443eea) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:28:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-2059820352',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=64e8066b-82ca-4278-ac3b-4ce6d3460710,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='90c0fbb4a6d84767b920564584715f9d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:28:47Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:28:47Z,use_quota=True,user_id='64090c8fd3394980ad5409adcde4986f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:47.502212 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9c0eb9c0-ab0f-44df-a956-e5b46b4dedd6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:47.506839 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9c0eb9c0-ab0f-44df-a956-e5b46b4dedd6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:47.511877 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d53b0654-70de-4960-841a-7381cd18eff6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:47.515843 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d53b0654-70de-4960-841a-7381cd18eff6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '64e8066b-82ca-4278-ac3b-4ce6d3460710', 'volume_name': 'volume-64e8066b-82ca-4278-ac3b-4ce6d3460710', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:47.520366 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3577a574-bc1a-4dcf-8086-9adb2ef2d85f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:47.524065 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3577a574-bc1a-4dcf-8086-9adb2ef2d85f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:47.528747 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (62de4acf-97d7-42f1-bd85-9285fd8b3d17) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:47.529761 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Volume 64e8066b-82ca-4278-ac3b-4ce6d3460710: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-64e8066b-82ca-4278-ac3b-4ce6d3460710', 'volume_size': 1} Sep 27 10:28:47.530136 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] creating volume 'volume-64e8066b-82ca-4278-ac3b-4ce6d3460710' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:28:47.530899 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:28:47.590634 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=16,cinder:INSERT=2,cinder:UPDATE=7 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:47.596229 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:28:47.596607 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:28:47.601602 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (62de4acf-97d7-42f1-bd85-9285fd8b3d17) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-64e8066b-82ca-4278-ac3b-4ce6d3460710', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:47.609000 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (88ce58ef-75ff-43ff-8d0b-a1f3cccb6892) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:47.621575 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Volume volume-64e8066b-82ca-4278-ac3b-4ce6d3460710 (64e8066b-82ca-4278-ac3b-4ce6d3460710): created successfully Sep 27 10:28:47.625456 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (88ce58ef-75ff-43ff-8d0b-a1f3cccb6892) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:47.631412 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Flow 'volume_create_manager' (1123d9e4-0421-4b79-afc0-28a8aeab1ea8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:28:47.642969 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-3c706de0-362d-4bbd-9a67-a1e88a10f780 tempest-AttachVolumeNegativeTest-1091756803 None] Created volume successfully. Sep 27 10:28:49.098830 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 req-2fa5db0a-9d0f-4f82-a88c-0b1ef4ddaade tempest-AttachVolumeNegativeTest-1091756803 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:28:50.031676 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 req-2fa5db0a-9d0f-4f82-a88c-0b1ef4ddaade tempest-AttachVolumeNegativeTest-1091756803 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.932s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:28:50.032673 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 req-2fa5db0a-9d0f-4f82-a88c-0b1ef4ddaade tempest-AttachVolumeNegativeTest-1091756803 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-64e8066b-82ca-4278-ac3b-4ce6d3460710', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '64e8066b-82ca-4278-ac3b-4ce6d3460710', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:28:50.065331 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 req-2fa5db0a-9d0f-4f82-a88c-0b1ef4ddaade tempest-AttachVolumeNegativeTest-1091756803 None] Connection info returned from driver {'name': 'volumes/volume-64e8066b-82ca-4278-ac3b-4ce6d3460710', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '64e8066b-82ca-4278-ac3b-4ce6d3460710', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '6e812462-cc6d-4bcb-b8c8-6c3411c5dac0'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:28:50.124925 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 req-2fa5db0a-9d0f-4f82-a88c-0b1ef4ddaade tempest-AttachVolumeNegativeTest-1091756803 None] attachment_update completed successfully. Sep 27 10:28:51.035364 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-6475cd4b-3005-4bba-98df-9adf739871ef tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['8c11bffe-ece9-49a6-91ff-2f7b5b200e78', '5f9b8f59-b0a8-4da7-9e61-ac9ee1779db7', '2784cdf9-6d94-4836-b71d-5b292ae13410', '1ce0facb-b4d2-4cd5-97b7-6e76f837477f'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:28:51.082691 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6475cd4b-3005-4bba-98df-9adf739871ef tempest-VolumesSnapshotTestJSON-1705236581 None] Delete snapshot completed successfully. Sep 27 10:28:51.088268 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6475cd4b-3005-4bba-98df-9adf739871ef tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-de9e38c1-d557-40ab-9b62-975274a202db-delete_snapshot" released by "delete_snapshot" :: held 30.220s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:28:52.805381 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c req-ec12943a-1c4a-4c42-b49a-c02f7481aeba tempest-VolumesSnapshotTestJSON-1705236581 None] Terminate volume connection completed successfully. Sep 27 10:28:52.806099 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c req-ec12943a-1c4a-4c42-b49a-c02f7481aeba tempest-VolumesSnapshotTestJSON-1705236581 None] Deleting attachment e2b72fd4-4c17-427a-a088-db2da7cc3246. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:28:57.533983 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:28:57.559891 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (711c32c6-7581-490a-adf3-39f9ee8e672c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:28:57.566585 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (63a38a64-c85c-4d88-8a04-0197320cc3b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:57.582646 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (63a38a64-c85c-4d88-8a04-0197320cc3b1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:28:57Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-2094046285',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a6deb5b5-2674-4a50-9e9e-c4149613d4cd,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='712c1c4c858b4b788a7703345716504b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:28:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:28:57Z,use_quota=True,user_id='584c395757a54d9b9a64dd399d8557b1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:57.587905 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4bbdfad1-5459-4ee2-8e4b-194af6918886) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:57.592399 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4bbdfad1-5459-4ee2-8e4b-194af6918886) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:57.597449 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (612164cd-25f4-449a-879d-bec5e9d3a1e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:57.601576 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (612164cd-25f4-449a-879d-bec5e9d3a1e1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a6deb5b5-2674-4a50-9e9e-c4149613d4cd', 'volume_name': 'volume-a6deb5b5-2674-4a50-9e9e-c4149613d4cd', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:57.606572 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (86d83e14-ad4b-4381-8b71-6224b34c34a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:57.610481 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (86d83e14-ad4b-4381-8b71-6224b34c34a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:57.617021 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7016170d-0f0d-4d3a-84ad-8ac24f9d1673) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:57.618225 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Volume a6deb5b5-2674-4a50-9e9e-c4149613d4cd: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a6deb5b5-2674-4a50-9e9e-c4149613d4cd', 'volume_size': 1} Sep 27 10:28:57.618737 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] creating volume 'volume-a6deb5b5-2674-4a50-9e9e-c4149613d4cd' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:28:57.620050 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:28:57.694355 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:28:57.694735 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:28:57.698339 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7016170d-0f0d-4d3a-84ad-8ac24f9d1673) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a6deb5b5-2674-4a50-9e9e-c4149613d4cd', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:57.702707 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f03a4071-0a50-4e5a-a01e-705142e7b6f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:28:57.712963 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Volume volume-a6deb5b5-2674-4a50-9e9e-c4149613d4cd (a6deb5b5-2674-4a50-9e9e-c4149613d4cd): created successfully Sep 27 10:28:57.716882 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f03a4071-0a50-4e5a-a01e-705142e7b6f9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:28:57.722310 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (711c32c6-7581-490a-adf3-39f9ee8e672c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:28:57.731356 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-610f1218-0eec-4640-8988-4108709a59fd tempest-VolumesSnapshotTestJSON-1705236581 None] Created volume successfully. Sep 27 10:28:58.746582 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-83811f6b-63c6-41e2-8fae-fc8303652a7a tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:28:59.555884 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-83811f6b-63c6-41e2-8fae-fc8303652a7a tempest-VolumesSnapshotTestJSON-1705236581 None] Create snapshot completed successfully Sep 27 10:29:00.164280 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:29:00.192458 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (480050c3-ad28-46d9-b1d4-dde2cd9d2eca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:29:00.198608 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7ac3c85b-532f-4ba5-8c23-9c1ecc689a0a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:00.216577 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7ac3c85b-532f-4ba5-8c23-9c1ecc689a0a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:29:00Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1305796165',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ed566acf-690d-4b8b-92e7-68bbc95c6320,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='712c1c4c858b4b788a7703345716504b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:29:00Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=034c82d5-5204-47b3-8719-e9c1c075328d,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:29:00Z,use_quota=True,user_id='584c395757a54d9b9a64dd399d8557b1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:00.222540 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (66886c96-b666-4fb5-bf64-02bd57de1a3f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:00.226593 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (66886c96-b666-4fb5-bf64-02bd57de1a3f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:00.231535 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (059dfcf8-02bd-44e9-98a4-aae812ea8628) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:00.235500 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (059dfcf8-02bd-44e9-98a4-aae812ea8628) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'ed566acf-690d-4b8b-92e7-68bbc95c6320', 'volume_name': 'volume-ed566acf-690d-4b8b-92e7-68bbc95c6320', 'volume_size': 2, 'snapshot_id': '034c82d5-5204-47b3-8719-e9c1c075328d'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:00.240402 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b4ae144f-b109-4aac-972f-29fcc5fe9058) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:00.244188 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b4ae144f-b109-4aac-972f-29fcc5fe9058) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:00.249180 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b3cdc1b4-4818-466b-babd-0d2a0a43c548) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:00.250640 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Volume ed566acf-690d-4b8b-92e7-68bbc95c6320: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-ed566acf-690d-4b8b-92e7-68bbc95c6320', 'volume_size': 2, 'snapshot_id': '034c82d5-5204-47b3-8719-e9c1c075328d'} Sep 27 10:29:00.275210 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] cloning volumes/volume-a6deb5b5-2674-4a50-9e9e-c4149613d4cd@snapshot-034c82d5-5204-47b3-8719-e9c1c075328d to volume-ed566acf-690d-4b8b-92e7-68bbc95c6320 {{(pid=138473) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 27 10:29:00.275931 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:00.321100 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:00.421596 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:29:00.421962 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:29:00.426571 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:00.549497 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b3cdc1b4-4818-466b-babd-0d2a0a43c548) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ed566acf-690d-4b8b-92e7-68bbc95c6320', 'volume_size': 2, 'snapshot_id': '034c82d5-5204-47b3-8719-e9c1c075328d'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:00.553958 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2790500c-9066-47e9-83fd-4217202a97d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:00.563737 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Volume volume-ed566acf-690d-4b8b-92e7-68bbc95c6320 (ed566acf-690d-4b8b-92e7-68bbc95c6320): created successfully Sep 27 10:29:00.567194 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2790500c-9066-47e9-83fd-4217202a97d7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:00.572464 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (480050c3-ad28-46d9-b1d4-dde2cd9d2eca) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:29:00.586050 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-44b789df-1583-419b-8d40-fbb8bb825951 tempest-VolumesSnapshotTestJSON-1705236581 None] Created volume successfully. Sep 27 10:29:01.297320 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-723a5576-a4de-4035-86d3-7fac4a524b73 tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-ed566acf-690d-4b8b-92e7-68bbc95c6320-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:29:01.316311 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-723a5576-a4de-4035-86d3-7fac4a524b73 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-ed566acf-690d-4b8b-92e7-68bbc95c6320-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:29:01.387168 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-723a5576-a4de-4035-86d3-7fac4a524b73 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:01.495453 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-723a5576-a4de-4035-86d3-7fac4a524b73 tempest-VolumesSnapshotTestJSON-1705236581 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:29:01.501947 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-723a5576-a4de-4035-86d3-7fac4a524b73 tempest-VolumesSnapshotTestJSON-1705236581 None] deleting rbd volume volume-ed566acf-690d-4b8b-92e7-68bbc95c6320 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:29:01.724237 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-723a5576-a4de-4035-86d3-7fac4a524b73 tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['2a51fa84-bc07-44b0-bd78-6985ac3a3e6d', '565a381d-3380-483a-bc26-ae8c21db67a2', '79b7a6b2-8769-43ee-8138-6cf706bc823f', '55fc80c2-b539-42b9-9f2c-6a8eb6bb0b34'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:29:01.772250 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-723a5576-a4de-4035-86d3-7fac4a524b73 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:01.800586 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-723a5576-a4de-4035-86d3-7fac4a524b73 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:01.835181 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-723a5576-a4de-4035-86d3-7fac4a524b73 tempest-VolumesSnapshotTestJSON-1705236581 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:29:01.840366 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-723a5576-a4de-4035-86d3-7fac4a524b73 tempest-VolumesSnapshotTestJSON-1705236581 None] Deleted volume successfully. Sep 27 10:29:01.856448 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-723a5576-a4de-4035-86d3-7fac4a524b73 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-ed566acf-690d-4b8b-92e7-68bbc95c6320-delete_volume" released by "delete_volume" :: held 0.540s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:29:02.798170 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:29:02.818617 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (1c0a1251-2f54-4a33-863e-a9251b94d89d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:29:02.825132 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8595f174-35f7-4ac9-95d8-90de4524bead) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:02.842695 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8595f174-35f7-4ac9-95d8-90de4524bead) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:29:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1902886697',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ca215b15-c6ad-4d97-814b-f8673cd6d387,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='712c1c4c858b4b788a7703345716504b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:29:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:29:03Z,use_quota=True,user_id='584c395757a54d9b9a64dd399d8557b1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:02.847869 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7c187fdf-a4e4-45bb-badd-32177cb03228) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:02.851819 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7c187fdf-a4e4-45bb-badd-32177cb03228) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:02.856701 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fd3ee710-d04f-4780-91c3-a39f7a55ce0e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:02.860622 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fd3ee710-d04f-4780-91c3-a39f7a55ce0e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ca215b15-c6ad-4d97-814b-f8673cd6d387', 'volume_name': 'volume-ca215b15-c6ad-4d97-814b-f8673cd6d387', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:02.865595 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (94fa3274-7c48-462c-9768-1578b4f626e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:02.869372 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (94fa3274-7c48-462c-9768-1578b4f626e1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:02.873884 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca5c10cc-e4cf-4b85-9fa1-66b8f443fba4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:02.874897 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Volume ca215b15-c6ad-4d97-814b-f8673cd6d387: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ca215b15-c6ad-4d97-814b-f8673cd6d387', 'volume_size': 1} Sep 27 10:29:02.875269 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] creating volume 'volume-ca215b15-c6ad-4d97-814b-f8673cd6d387' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:29:02.876056 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:02.929321 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:29:02.929605 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:29:02.933288 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca5c10cc-e4cf-4b85-9fa1-66b8f443fba4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ca215b15-c6ad-4d97-814b-f8673cd6d387', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:02.937666 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2042eca4-2f49-4476-b316-decc26b8f75f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:02.949726 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Volume volume-ca215b15-c6ad-4d97-814b-f8673cd6d387 (ca215b15-c6ad-4d97-814b-f8673cd6d387): created successfully Sep 27 10:29:02.956969 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2042eca4-2f49-4476-b316-decc26b8f75f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:02.962800 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (1c0a1251-2f54-4a33-863e-a9251b94d89d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:29:02.974384 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-73409b94-6f28-4f79-8b08-b5c1bcc2a766 tempest-VolumesSnapshotTestJSON-1705236581 None] Created volume successfully. Sep 27 10:29:04.015373 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ed46a436-22bf-46f1-8d47-cd32d771342c tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:04.662391 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ed46a436-22bf-46f1-8d47-cd32d771342c tempest-VolumesSnapshotTestJSON-1705236581 None] Create snapshot completed successfully Sep 27 10:29:05.644404 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=68,cinder:UPDATE=24,cinder:INSERT=4,cinder:DELETE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:05.660971 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:29:05.695268 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (6f865d47-e939-429e-bd41-6f845436b913) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:29:05.704234 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2aaef448-4001-4bee-a2f0-ead9aca4a112) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:05.769479 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2aaef448-4001-4bee-a2f0-ead9aca4a112) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:29:05Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1999530331',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d33f88b2-e199-4ec0-b910-90b17e3ef1c0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='712c1c4c858b4b788a7703345716504b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:29:06Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=aa9db8d0-bd83-46b3-a695-91b82bd74ed2,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:29:06Z,use_quota=True,user_id='584c395757a54d9b9a64dd399d8557b1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:05.769651 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1ce84e15-0c2e-4f44-85af-557c111298d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:05.770143 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1ce84e15-0c2e-4f44-85af-557c111298d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:05.775135 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (101fbbf1-6c99-44ea-b8ab-9101edc1ca95) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:05.779243 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (101fbbf1-6c99-44ea-b8ab-9101edc1ca95) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'd33f88b2-e199-4ec0-b910-90b17e3ef1c0', 'volume_name': 'volume-d33f88b2-e199-4ec0-b910-90b17e3ef1c0', 'volume_size': 1, 'snapshot_id': 'aa9db8d0-bd83-46b3-a695-91b82bd74ed2'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:05.784308 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7d76c573-97ea-4535-a158-0f1172348ffd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:05.788354 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7d76c573-97ea-4535-a158-0f1172348ffd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:05.792958 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (817383f0-8ea7-4b78-a553-0dfb8529afe8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:05.794248 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Volume d33f88b2-e199-4ec0-b910-90b17e3ef1c0: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-d33f88b2-e199-4ec0-b910-90b17e3ef1c0', 'volume_size': 1, 'snapshot_id': 'aa9db8d0-bd83-46b3-a695-91b82bd74ed2'} Sep 27 10:29:05.845726 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] cloning volumes/volume-ca215b15-c6ad-4d97-814b-f8673cd6d387@snapshot-aa9db8d0-bd83-46b3-a695-91b82bd74ed2 to volume-d33f88b2-e199-4ec0-b910-90b17e3ef1c0 {{(pid=138473) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 27 10:29:05.846755 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:05.890133 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:06.049489 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:29:06.050130 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:29:06.072171 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (817383f0-8ea7-4b78-a553-0dfb8529afe8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d33f88b2-e199-4ec0-b910-90b17e3ef1c0', 'volume_size': 1, 'snapshot_id': 'aa9db8d0-bd83-46b3-a695-91b82bd74ed2'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:06.076933 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1582c43b-2e61-4b40-bf92-1f15fe8b5d13) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:29:06.091807 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Volume volume-d33f88b2-e199-4ec0-b910-90b17e3ef1c0 (d33f88b2-e199-4ec0-b910-90b17e3ef1c0): created successfully Sep 27 10:29:06.096953 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1582c43b-2e61-4b40-bf92-1f15fe8b5d13) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:29:06.103787 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Flow 'volume_create_manager' (6f865d47-e939-429e-bd41-6f845436b913) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:29:06.121570 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-aeeba0ac-cdde-4efe-8001-63d804c1c06c tempest-VolumesSnapshotTestJSON-1705236581 None] Created volume successfully. Sep 27 10:29:06.126638 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:DELETE=3,cinder:SELECT=67,cinder:UPDATE=26,cinder:INSERT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:06.767586 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-99f7a639-e0a7-4df0-8236-fcb1fe1c07e7 tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-d33f88b2-e199-4ec0-b910-90b17e3ef1c0-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:29:06.774794 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-99f7a639-e0a7-4df0-8236-fcb1fe1c07e7 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-d33f88b2-e199-4ec0-b910-90b17e3ef1c0-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:29:06.842003 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-99f7a639-e0a7-4df0-8236-fcb1fe1c07e7 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:06.913104 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-99f7a639-e0a7-4df0-8236-fcb1fe1c07e7 tempest-VolumesSnapshotTestJSON-1705236581 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:29:06.924040 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-99f7a639-e0a7-4df0-8236-fcb1fe1c07e7 tempest-VolumesSnapshotTestJSON-1705236581 None] deleting rbd volume volume-d33f88b2-e199-4ec0-b910-90b17e3ef1c0 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:29:07.305878 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-99f7a639-e0a7-4df0-8236-fcb1fe1c07e7 tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['08bb8f6e-c5ec-452d-9ff0-2a339194f828', '8a4417c7-5a88-48b6-a8bf-043e6e55f919', '7f524c98-a210-4dcf-ad70-2e27334c45bd', '3db94519-821f-499c-a1b3-dbaee1582360'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:29:07.358645 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-99f7a639-e0a7-4df0-8236-fcb1fe1c07e7 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:07.407914 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-99f7a639-e0a7-4df0-8236-fcb1fe1c07e7 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:07.459538 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-99f7a639-e0a7-4df0-8236-fcb1fe1c07e7 tempest-VolumesSnapshotTestJSON-1705236581 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:29:07.465597 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-99f7a639-e0a7-4df0-8236-fcb1fe1c07e7 tempest-VolumesSnapshotTestJSON-1705236581 None] Deleted volume successfully. Sep 27 10:29:07.481352 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-99f7a639-e0a7-4df0-8236-fcb1fe1c07e7 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-d33f88b2-e199-4ec0-b910-90b17e3ef1c0-delete_volume" released by "delete_volume" :: held 0.706s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:29:07.988950 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6bd5bda3-377e-4a7c-a170-83c5883a5183 tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-aa9db8d0-bd83-46b3-a695-91b82bd74ed2-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:29:08.011954 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6bd5bda3-377e-4a7c-a170-83c5883a5183 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-aa9db8d0-bd83-46b3-a695-91b82bd74ed2-delete_snapshot" acquired by "delete_snapshot" :: waited 0.023s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:29:08.014273 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6bd5bda3-377e-4a7c-a170-83c5883a5183 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:08.802303 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-6bd5bda3-377e-4a7c-a170-83c5883a5183 tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['43b60986-2afd-4bd9-8b42-bcbbc43be9f1', '7f6a2571-64d2-4d4c-b42a-2f0c1fb6c811', '8d05e5fe-f3a4-4c6c-a15b-6ba18ee21aa4', '1dca4c1b-b6ef-4068-929d-c085cce65d1b'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:29:08.864812 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6bd5bda3-377e-4a7c-a170-83c5883a5183 tempest-VolumesSnapshotTestJSON-1705236581 None] Delete snapshot completed successfully. Sep 27 10:29:08.871217 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6bd5bda3-377e-4a7c-a170-83c5883a5183 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-aa9db8d0-bd83-46b3-a695-91b82bd74ed2-delete_snapshot" released by "delete_snapshot" :: held 0.859s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:29:09.161389 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-ca215b15-c6ad-4d97-814b-f8673cd6d387-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:29:09.184688 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-ca215b15-c6ad-4d97-814b-f8673cd6d387-delete_volume" acquired by "delete_volume" :: waited 0.023s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:29:09.260737 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:09.362927 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:29:09.363389 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] Volume volume-ca215b15-c6ad-4d97-814b-f8673cd6d387 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:29:09.390247 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] deleting rbd volume volume-ca215b15-c6ad-4d97-814b-f8673cd6d387 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:29:09.788518 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['a72c1f45-8a2d-4bf2-80dc-8b93e214dda5', '14207b8e-2b66-4b28-9965-a83113343c1c', '9e6f46ee-6ba8-40fd-adfd-a7098e3e523d', 'a864b056-a0df-4c10-9e1c-fa2f1137cbf4'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:29:09.870716 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:09.929296 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:09.985680 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:29:09.994967 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] Deleted volume successfully. Sep 27 10:29:10.015448 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6c3ed54b-0a91-47d1-b141-c5119014e557 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-ca215b15-c6ad-4d97-814b-f8673cd6d387-delete_volume" released by "delete_volume" :: held 0.831s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:29:10.422161 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0ac477d1-a18d-4357-8233-c03ae0e5f9fa tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-034c82d5-5204-47b3-8719-e9c1c075328d-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:29:10.435411 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0ac477d1-a18d-4357-8233-c03ae0e5f9fa tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-034c82d5-5204-47b3-8719-e9c1c075328d-delete_snapshot" acquired by "delete_snapshot" :: waited 0.013s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:29:10.438541 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0ac477d1-a18d-4357-8233-c03ae0e5f9fa tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:10.766697 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-0ac477d1-a18d-4357-8233-c03ae0e5f9fa tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['514d4d6e-1833-44dc-81af-0dcf55d9b2f8', '9f618244-31bf-41f3-a138-bf094bcac53f', '7fb74c12-95ac-4248-88a5-fb1e24907c61', '1a1d425a-b810-4ffa-b0de-f68d2a7d9009'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:29:10.811459 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0ac477d1-a18d-4357-8233-c03ae0e5f9fa tempest-VolumesSnapshotTestJSON-1705236581 None] Delete snapshot completed successfully. Sep 27 10:29:10.817338 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0ac477d1-a18d-4357-8233-c03ae0e5f9fa tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-034c82d5-5204-47b3-8719-e9c1c075328d-delete_snapshot" released by "delete_snapshot" :: held 0.382s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:29:11.570931 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-a6deb5b5-2674-4a50-9e9e-c4149613d4cd-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:29:11.602192 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-a6deb5b5-2674-4a50-9e9e-c4149613d4cd-delete_volume" acquired by "delete_volume" :: waited 0.031s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:29:11.690228 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:11.761517 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:29:11.762097 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] Volume volume-a6deb5b5-2674-4a50-9e9e-c4149613d4cd is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:29:11.773319 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] deleting rbd volume volume-a6deb5b5-2674-4a50-9e9e-c4149613d4cd {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:29:12.201211 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['9ab4e1fe-b755-4f65-bfe9-ece1fae2e277', '8078a1b2-eee4-4b46-9d7f-366fb7cb8862', '5fde0b8b-5a1e-41c5-8a00-f3789c3247da', '160b4c5f-2762-494e-9834-62ce0473044f'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:29:12.210614 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=60,cinder:INSERT=6,cinder:UPDATE=33,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:12.243398 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=53,cinder:UPDATE=31,cinder:DELETE=2,cinder:INSERT=14 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:12.282395 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:12.349248 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:12.408752 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:29:12.413168 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] Deleted volume successfully. Sep 27 10:29:12.429266 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5c11639a-618a-4645-91f8-3a46d5bb4e9e tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-a6deb5b5-2674-4a50-9e9e-c4149613d4cd-delete_volume" released by "delete_volume" :: held 0.827s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:29:13.336701 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:29:13.363112 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5-delete_volume" acquired by "delete_volume" :: waited 0.026s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:29:13.412711 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:13.516507 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:29:13.530169 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] deleting rbd volume volume-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:29:13.778624 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:29:13.813380 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] deleting parent snapshot volume-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:29:14.853524 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['512a161d-3ae4-4529-85ae-814be838cefa', '728aa38d-dcf8-4cbd-ada2-2987e966175c', '7bcbeff5-78cb-41a3-9aed-916acb7f8636', '80f0d5a7-f936-4970-aa45-287fa1cfab91'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:29:14.935935 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:15.014703 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:15.093373 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:29:15.103425 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] Deleted volume successfully. Sep 27 10:29:15.123066 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b4f1b3f8-b635-4bdf-b931-f9129bc7c5f7 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-1962f1c2-cd1b-44c0-ab12-4f391cd8acf5-delete_volume" released by "delete_volume" :: held 1.760s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:29:15.636940 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-ffde3e5f-d086-4a73-80ed-3cdd02202dc3 tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-65d29fc0-a3b9-4942-8752-0c17e1a6c717-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:29:15.650813 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-ffde3e5f-d086-4a73-80ed-3cdd02202dc3 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-65d29fc0-a3b9-4942-8752-0c17e1a6c717-delete_snapshot" acquired by "delete_snapshot" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:29:15.652539 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ffde3e5f-d086-4a73-80ed-3cdd02202dc3 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:16.925402 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-ffde3e5f-d086-4a73-80ed-3cdd02202dc3 tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['1679b8e7-d5cd-4266-ba7e-383f01638a66', 'b9c9ed4f-f816-423d-ba96-a88ba12ff92f', '996af910-431f-4421-b495-943922f74cff', 'f1bff8d2-73eb-488b-af10-c1e3c8e22d1f'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:29:17.005291 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ffde3e5f-d086-4a73-80ed-3cdd02202dc3 tempest-VolumesSnapshotTestJSON-1705236581 None] Delete snapshot completed successfully. Sep 27 10:29:17.011099 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-ffde3e5f-d086-4a73-80ed-3cdd02202dc3 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-65d29fc0-a3b9-4942-8752-0c17e1a6c717-delete_snapshot" released by "delete_snapshot" :: held 1.363s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:29:17.888451 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] Acquiring lock "cinder-cd03ad33-8f4c-4084-8b36-72d1fe84c526-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:29:17.902676 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-cd03ad33-8f4c-4084-8b36-72d1fe84c526-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:29:17.961852 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:18.030453 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:29:18.031120 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] Volume volume-cd03ad33-8f4c-4084-8b36-72d1fe84c526 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:29:18.038768 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] deleting rbd volume volume-cd03ad33-8f4c-4084-8b36-72d1fe84c526 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:29:18.305970 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] Created reservations ['2bd3683f-07bb-4a27-b744-f071c52e69fa', '6ba2e53b-5caa-48a6-8716-f735a5a372b5', '17c086fd-637b-4f41-8d78-d14cc14a8285', '2883e7cc-895a-46fa-bece-e8b0e552b422'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:29:18.352543 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:18.384363 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:18.414817 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:29:18.419315 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] Deleted volume successfully. Sep 27 10:29:18.433924 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3d85d4e5-7a27-42c0-93fc-d3d312e44057 tempest-VolumesSnapshotTestJSON-1705236581 None] Lock "cinder-cd03ad33-8f4c-4084-8b36-72d1fe84c526-delete_volume" released by "delete_volume" :: held 0.534s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:29:29.649368 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 req-75c03554-4342-4804-ab1d-efb36d931904 tempest-TestVolumeBootPattern-4600220 None] Terminate volume connection completed successfully. Sep 27 10:29:29.650091 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 req-75c03554-4342-4804-ab1d-efb36d931904 tempest-TestVolumeBootPattern-4600220 None] Deleting attachment 7d7112d2-60a1-4cff-8c78-03eecfe73a4f. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:29:29.791106 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:29:29.792561 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:29.847605 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:29:29.910646 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:29:33.524605 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 req-f8f8488c-0a42-4d0b-9e8c-0c66ed3d0757 tempest-TestVolumeBootPattern-4600220 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:29:34.282735 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 req-f8f8488c-0a42-4d0b-9e8c-0c66ed3d0757 tempest-TestVolumeBootPattern-4600220 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.758s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:29:34.283665 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 req-f8f8488c-0a42-4d0b-9e8c-0c66ed3d0757 tempest-TestVolumeBootPattern-4600220 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-8489f678-48b9-47ca-8f5e-397c453365e4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '8489f678-48b9-47ca-8f5e-397c453365e4', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:29:34.323382 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 req-f8f8488c-0a42-4d0b-9e8c-0c66ed3d0757 tempest-TestVolumeBootPattern-4600220 None] Connection info returned from driver {'name': 'volumes/volume-8489f678-48b9-47ca-8f5e-397c453365e4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '8489f678-48b9-47ca-8f5e-397c453365e4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '2700969a-6927-4e00-88af-2a1c3977077a'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:29:34.399540 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 req-f8f8488c-0a42-4d0b-9e8c-0c66ed3d0757 tempest-TestVolumeBootPattern-4600220 None] attachment_update completed successfully. Sep 27 10:29:47.653273 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:DELETE=3,cinder:SELECT=44,cinder:UPDATE=27,cinder:INSERT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:47.654919 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=61,cinder:UPDATE=30,cinder:INSERT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:03.856544 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:30:03.876588 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Flow 'volume_create_manager' (d65975f4-272a-4979-97c5-567663be2a6b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:30:03.883230 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (22a0f7e8-2465-4adf-b8bd-2580c7fbfd77) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:03.899825 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (22a0f7e8-2465-4adf-b8bd-2580c7fbfd77) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:30:04Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerRescueNegativeTestJSON-volume-839107530',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c4b02a63-754a-4f44-ab9f-69541e872afc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='40247edd5a444fd8b1c937c13d39393a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:30:04Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:30:04Z,use_quota=True,user_id='6771df45b4ee4426b2f76a09eab2998a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:03.905033 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0f28b6c9-2066-460c-9c8a-8f582e869db1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:03.909122 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0f28b6c9-2066-460c-9c8a-8f582e869db1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:03.914004 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (46849599-8f23-474c-83f4-9fa3f0c9fda9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:03.917974 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (46849599-8f23-474c-83f4-9fa3f0c9fda9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c4b02a63-754a-4f44-ab9f-69541e872afc', 'volume_name': 'volume-c4b02a63-754a-4f44-ab9f-69541e872afc', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:03.922726 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9bd917fe-e625-4932-913f-722c80dd8504) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:03.926456 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9bd917fe-e625-4932-913f-722c80dd8504) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:03.930954 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4175ab7e-ada8-4715-bf01-ccfe4d9fb2f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:03.932018 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Volume c4b02a63-754a-4f44-ab9f-69541e872afc: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c4b02a63-754a-4f44-ab9f-69541e872afc', 'volume_size': 1} Sep 27 10:30:03.932423 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] creating volume 'volume-c4b02a63-754a-4f44-ab9f-69541e872afc' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:30:03.933349 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:30:03.989250 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:30:03.989701 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:30:03.993614 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4175ab7e-ada8-4715-bf01-ccfe4d9fb2f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c4b02a63-754a-4f44-ab9f-69541e872afc', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:03.998069 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (335d475c-ec28-46a9-83ab-c6d06c47c762) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:04.010209 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Volume volume-c4b02a63-754a-4f44-ab9f-69541e872afc (c4b02a63-754a-4f44-ab9f-69541e872afc): created successfully Sep 27 10:30:04.013780 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (335d475c-ec28-46a9-83ab-c6d06c47c762) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:04.019088 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Flow 'volume_create_manager' (d65975f4-272a-4979-97c5-567663be2a6b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:30:04.029315 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-4ee0e763-3075-436c-8cc9-c9acfa2d4a43 tempest-ServerRescueNegativeTestJSON-1875151076 None] Created volume successfully. Sep 27 10:30:27.496813 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=10,cinder:DELETE=1,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:27.498854 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=10,cinder:UPDATE=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:27.514737 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:30:27.533763 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Flow 'volume_create_manager' (69dbaf30-c162-468a-81fd-7ee1c01916c4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:30:27.539873 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (08534b20-5a28-469f-8cb9-1957252a6801) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:27.555763 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (08534b20-5a28-469f-8cb9-1957252a6801) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:30:27Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerRescueNegativeTestJSON-volume-236893746',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='40247edd5a444fd8b1c937c13d39393a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:30:27Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:30:27Z,use_quota=True,user_id='6771df45b4ee4426b2f76a09eab2998a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:27.560796 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a049f5d6-f93b-4247-b8d5-c1938dd98f01) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:27.565180 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a049f5d6-f93b-4247-b8d5-c1938dd98f01) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:27.569980 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a0d2f847-7c4f-4200-aadf-55c3f6f78b2c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:27.573813 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a0d2f847-7c4f-4200-aadf-55c3f6f78b2c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3', 'volume_name': 'volume-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:27.578435 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1be92d93-9936-4086-bdca-1f64553d1cca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:27.582416 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1be92d93-9936-4086-bdca-1f64553d1cca) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:27.586837 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4be2ef38-bdc3-47e4-8e64-2b211d9ccf2e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:27.587895 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Volume a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3', 'volume_size': 1} Sep 27 10:30:27.588282 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] creating volume 'volume-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:30:27.588925 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:30:27.657012 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:30:27.657387 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:30:27.661042 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4be2ef38-bdc3-47e4-8e64-2b211d9ccf2e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:27.665466 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (46da6aea-9e8f-46e9-86b6-43138688ba23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:30:27.681971 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Volume volume-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3 (a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3): created successfully Sep 27 10:30:27.685817 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (46da6aea-9e8f-46e9-86b6-43138688ba23) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:30:27.691216 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Flow 'volume_create_manager' (69dbaf30-c162-468a-81fd-7ee1c01916c4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:30:27.703087 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b699e7ad-d5b1-45bb-8830-b6fca37830f5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Created volume successfully. Sep 27 10:30:29.798485 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:30:29.799807 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:30:29.829104 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:30:29.863303 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:30:30.008603 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-74ee7503-b943-44ba-97c1-16ce9da8462d req-66b698eb-95d0-402d-95d6-c6667d318a2b tempest-AttachVolumeNegativeTest-1091756803 None] Terminate volume connection completed successfully. Sep 27 10:30:30.009079 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-74ee7503-b943-44ba-97c1-16ce9da8462d req-66b698eb-95d0-402d-95d6-c6667d318a2b tempest-AttachVolumeNegativeTest-1091756803 None] Deleting attachment 6e812462-cc6d-4bcb-b8c8-6c3411c5dac0. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:30:57.647095 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=11,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:57.649185 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=10,cinder:UPDATE=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:26.719463 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6cbfd9aa-e251-4dc6-98ba-eff991716cb0 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:31:27.669557 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=9,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:27.670550 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=10,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:27.727565 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6cbfd9aa-e251-4dc6-98ba-eff991716cb0 tempest-TestVolumeBootPattern-4600220 None] Create snapshot completed successfully Sep 27 10:31:29.188555 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:31:29.215881 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (094227f5-e567-4e5a-8fe9-e65a2bbf95fa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:31:29.222330 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (247a72b2-d4c8-43d2-8811-af2e3e8a57e7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:29.238850 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (247a72b2-d4c8-43d2-8811-af2e3e8a57e7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:31:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-443129882',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c2c6c4b3-ba4e-4994-8674-d39521abb223,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='64f8fe93425d4e97b1ae5661172f4846',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:31:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=257d6e25-918d-4ac2-b78c-03b14811a9f2,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:31:29Z,use_quota=True,user_id='178956f6c8154a06a0710b295aff3086',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:29.244203 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e09da41a-658a-4926-bc62-c306c06df105) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:29.248117 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e09da41a-658a-4926-bc62-c306c06df105) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:29.252920 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f5e5cc1c-dbb7-4739-8b39-632f31e3130e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:29.256738 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f5e5cc1c-dbb7-4739-8b39-632f31e3130e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'c2c6c4b3-ba4e-4994-8674-d39521abb223', 'volume_name': 'volume-c2c6c4b3-ba4e-4994-8674-d39521abb223', 'volume_size': 1, 'snapshot_id': '257d6e25-918d-4ac2-b78c-03b14811a9f2'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:29.261502 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ce42788d-2f7c-4176-b28e-36f9950c62d3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:29.265272 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ce42788d-2f7c-4176-b28e-36f9950c62d3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:29.269677 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d6dcffd8-7e82-4146-90bc-c674a4081c8b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:29.270702 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Volume c2c6c4b3-ba4e-4994-8674-d39521abb223: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-c2c6c4b3-ba4e-4994-8674-d39521abb223', 'volume_size': 1, 'snapshot_id': '257d6e25-918d-4ac2-b78c-03b14811a9f2'} Sep 27 10:31:29.294569 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] cloning volumes/volume-8489f678-48b9-47ca-8f5e-397c453365e4@snapshot-257d6e25-918d-4ac2-b78c-03b14811a9f2 to volume-c2c6c4b3-ba4e-4994-8674-d39521abb223 {{(pid=138473) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 27 10:31:29.295191 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:31:29.360052 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:31:29.487084 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:31:29.487349 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:31:29.507723 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Marking volume c2c6c4b3-ba4e-4994-8674-d39521abb223 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:31:29.519122 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Copying metadata from snapshot 257d6e25-918d-4ac2-b78c-03b14811a9f2 to c2c6c4b3-ba4e-4994-8674-d39521abb223. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:431}} Sep 27 10:31:29.550077 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d6dcffd8-7e82-4146-90bc-c674a4081c8b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c2c6c4b3-ba4e-4994-8674-d39521abb223', 'volume_size': 1, 'snapshot_id': '257d6e25-918d-4ac2-b78c-03b14811a9f2'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:29.554492 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (69d26fde-f4dd-40f6-aa03-1736384ae0bd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:29.564727 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Volume volume-c2c6c4b3-ba4e-4994-8674-d39521abb223 (c2c6c4b3-ba4e-4994-8674-d39521abb223): created successfully Sep 27 10:31:29.568050 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (69d26fde-f4dd-40f6-aa03-1736384ae0bd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:29.573295 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Flow 'volume_create_manager' (094227f5-e567-4e5a-8fe9-e65a2bbf95fa) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:31:29.587133 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-95c101c4-e058-4f78-8ee6-095b8abe06ba tempest-TestVolumeBootPattern-4600220 None] Created volume successfully. Sep 27 10:31:29.800091 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:31:29.801007 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:31:29.831999 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:31:29.857630 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:31:30.870356 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:31:30.889275 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Flow 'volume_create_manager' (c41e7da6-d6e7-4203-abb3-7a56234489f8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:31:30.895484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e9e2909c-869b-49fd-8859-670b155d36fe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:30.911702 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e9e2909c-869b-49fd-8859-670b155d36fe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:31:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-212416472',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4f4a8bff-8624-4fb4-b665-2f8273aaf6ea,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='90c0fbb4a6d84767b920564584715f9d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:31:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:31:31Z,use_quota=True,user_id='64090c8fd3394980ad5409adcde4986f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:30.917161 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (46054c6c-3d8f-4a9b-bef7-71bfdee4eab0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:30.921158 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (46054c6c-3d8f-4a9b-bef7-71bfdee4eab0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:30.925998 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff67e332-44a6-45a5-8216-680cfee8392e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:30.929906 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff67e332-44a6-45a5-8216-680cfee8392e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4f4a8bff-8624-4fb4-b665-2f8273aaf6ea', 'volume_name': 'volume-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:30.935198 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e88be41e-890c-4492-93e1-f6b396eb5374) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:30.941194 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e88be41e-890c-4492-93e1-f6b396eb5374) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:30.945906 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6b935cb5-fd28-47d6-a290-2e5c2b005f27) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:30.947174 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Volume 4f4a8bff-8624-4fb4-b665-2f8273aaf6ea: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea', 'volume_size': 1} Sep 27 10:31:30.947432 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] creating volume 'volume-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:31:30.948090 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:31:31.003608 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:31:31.003987 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:31:31.008035 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6b935cb5-fd28-47d6-a290-2e5c2b005f27) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:31.013411 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (314aa228-db96-4bd0-bb70-795ef387e855) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:31:31.023243 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Volume volume-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea (4f4a8bff-8624-4fb4-b665-2f8273aaf6ea): created successfully Sep 27 10:31:31.027136 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (314aa228-db96-4bd0-bb70-795ef387e855) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:31:31.032484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Flow 'volume_create_manager' (c41e7da6-d6e7-4203-abb3-7a56234489f8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:31:31.042266 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-178af03e-bdcc-46a7-a6df-ae90e03f7896 tempest-AttachVolumeNegativeTest-1091756803 None] Created volume successfully. Sep 27 10:31:32.481773 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-973b0829-4c87-4370-a265-2e1549ef7e54 req-a6bf54f0-5325-47a0-aa8a-5fa60dbdd8cb tempest-TestVolumeBootPattern-4600220 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:31:32.548409 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 req-c7050063-b474-427e-885c-0982deb4cb8b tempest-AttachVolumeNegativeTest-1091756803 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:31:33.287813 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-973b0829-4c87-4370-a265-2e1549ef7e54 req-a6bf54f0-5325-47a0-aa8a-5fa60dbdd8cb tempest-TestVolumeBootPattern-4600220 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.806s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:31:33.288920 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-973b0829-4c87-4370-a265-2e1549ef7e54 req-a6bf54f0-5325-47a0-aa8a-5fa60dbdd8cb tempest-TestVolumeBootPattern-4600220 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-c2c6c4b3-ba4e-4994-8674-d39521abb223', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'c2c6c4b3-ba4e-4994-8674-d39521abb223', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:31:33.323860 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-973b0829-4c87-4370-a265-2e1549ef7e54 req-a6bf54f0-5325-47a0-aa8a-5fa60dbdd8cb tempest-TestVolumeBootPattern-4600220 None] Connection info returned from driver {'name': 'volumes/volume-c2c6c4b3-ba4e-4994-8674-d39521abb223', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'c2c6c4b3-ba4e-4994-8674-d39521abb223', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '50e881dc-ef77-42cc-8ab3-8591492443d4'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:31:33.376650 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 req-c7050063-b474-427e-885c-0982deb4cb8b tempest-AttachVolumeNegativeTest-1091756803 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.829s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:31:33.377580 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 req-c7050063-b474-427e-885c-0982deb4cb8b tempest-AttachVolumeNegativeTest-1091756803 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '4f4a8bff-8624-4fb4-b665-2f8273aaf6ea', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:31:33.406929 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-973b0829-4c87-4370-a265-2e1549ef7e54 req-a6bf54f0-5325-47a0-aa8a-5fa60dbdd8cb tempest-TestVolumeBootPattern-4600220 None] attachment_update completed successfully. Sep 27 10:31:33.417196 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 req-c7050063-b474-427e-885c-0982deb4cb8b tempest-AttachVolumeNegativeTest-1091756803 None] Connection info returned from driver {'name': 'volumes/volume-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '4f4a8bff-8624-4fb4-b665-2f8273aaf6ea', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'f7189a36-e1e5-415c-aebf-b23b18e754b9'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:31:33.505262 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 req-c7050063-b474-427e-885c-0982deb4cb8b tempest-AttachVolumeNegativeTest-1091756803 None] attachment_update completed successfully. Sep 27 10:31:37.009143 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-3c624d48-a4e0-4b84-852b-f0f554689d4a req-60873c4c-6cdb-49b7-9b49-69b37d24a360 tempest-AttachVolumeNegativeTest-1091756803 None] Terminate volume connection completed successfully. Sep 27 10:31:37.009761 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-3c624d48-a4e0-4b84-852b-f0f554689d4a req-60873c4c-6cdb-49b7-9b49-69b37d24a360 tempest-AttachVolumeNegativeTest-1091756803 None] Deleting attachment f7189a36-e1e5-415c-aebf-b23b18e754b9. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:32:04.355589 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=50,cinder:INSERT=9,cinder:UPDATE=6,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:04.357568 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:INSERT=15,cinder:UPDATE=11,cinder:DELETE=1,cinder:SELECT=46 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:04.370627 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 req-bb2c15fb-64e4-4836-af42-8de387b6f860 tempest-ServerRescueNegativeTestJSON-1875151076 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:32:05.111185 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 req-bb2c15fb-64e4-4836-af42-8de387b6f860 tempest-ServerRescueNegativeTestJSON-1875151076 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.740s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:32:05.112627 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 req-bb2c15fb-64e4-4836-af42-8de387b6f860 tempest-ServerRescueNegativeTestJSON-1875151076 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:32:05.148567 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 req-bb2c15fb-64e4-4836-af42-8de387b6f860 tempest-ServerRescueNegativeTestJSON-1875151076 None] Connection info returned from driver {'name': 'volumes/volume-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '040d0e4e-a4ac-4258-a0c8-dce7cd95af51'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:32:05.218900 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 req-bb2c15fb-64e4-4836-af42-8de387b6f860 tempest-ServerRescueNegativeTestJSON-1875151076 None] attachment_update completed successfully. Sep 27 10:32:29.803072 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:32:29.804014 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:32:29.832180 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:32:29.858795 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:32:37.633079 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=11,cinder:UPDATE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:37.636094 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=18,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:13.971656 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:13.973294 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:13.989664 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:33:14.009238 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Flow 'volume_create_manager' (023ac055-3125-4bdb-b82f-f6eeacc32c59) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:33:14.015585 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1246ef46-1ac8-4be4-b5c8-4a79008307bd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:14.031792 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1246ef46-1ac8-4be4-b5c8-4a79008307bd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:33:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-225395641',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=80fa90a7-dc37-41b3-9de3-7d7c37cc72e1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='90c0fbb4a6d84767b920564584715f9d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:33:14Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:33:14Z,use_quota=True,user_id='64090c8fd3394980ad5409adcde4986f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:14.036830 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (038bad31-c494-40cb-9688-bf8eabb7a3b0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:14.040710 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (038bad31-c494-40cb-9688-bf8eabb7a3b0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:14.045769 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (934a12fd-bae7-464b-9a64-92e9056aafa7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:14.049614 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (934a12fd-bae7-464b-9a64-92e9056aafa7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '80fa90a7-dc37-41b3-9de3-7d7c37cc72e1', 'volume_name': 'volume-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:14.054200 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (05fa0d4e-81be-452e-8771-79852d5e9c25) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:14.058159 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (05fa0d4e-81be-452e-8771-79852d5e9c25) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:14.062471 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (52605636-922b-4795-ba82-572476f21bb8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:14.063501 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Volume 80fa90a7-dc37-41b3-9de3-7d7c37cc72e1: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1', 'volume_size': 1} Sep 27 10:33:14.063922 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] creating volume 'volume-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:33:14.064551 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:14.120704 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:33:14.121111 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:33:14.124723 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (52605636-922b-4795-ba82-572476f21bb8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:14.129147 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3d1496ff-1562-4bd2-a3dc-7774f4c47a86) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:14.140764 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Volume volume-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1 (80fa90a7-dc37-41b3-9de3-7d7c37cc72e1): created successfully Sep 27 10:33:14.144386 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3d1496ff-1562-4bd2-a3dc-7774f4c47a86) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:14.149829 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Flow 'volume_create_manager' (023ac055-3125-4bdb-b82f-f6eeacc32c59) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:33:14.160432 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-dc2166d6-0361-4d8c-85b2-2b9eb599ae4f tempest-AttachVolumeNegativeTest-1091756803 None] Created volume successfully. Sep 27 10:33:15.510163 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-e3db5609-ad35-4c28-ace8-3b6541f6b176 req-0ec9ba36-a106-4ac1-94cf-a244d69c8b11 tempest-AttachVolumeNegativeTest-1091756803 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:33:16.251784 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-e3db5609-ad35-4c28-ace8-3b6541f6b176 req-0ec9ba36-a106-4ac1-94cf-a244d69c8b11 tempest-AttachVolumeNegativeTest-1091756803 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.741s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:33:16.252630 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-e3db5609-ad35-4c28-ace8-3b6541f6b176 req-0ec9ba36-a106-4ac1-94cf-a244d69c8b11 tempest-AttachVolumeNegativeTest-1091756803 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '80fa90a7-dc37-41b3-9de3-7d7c37cc72e1', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:33:16.280993 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-e3db5609-ad35-4c28-ace8-3b6541f6b176 req-0ec9ba36-a106-4ac1-94cf-a244d69c8b11 tempest-AttachVolumeNegativeTest-1091756803 None] Connection info returned from driver {'name': 'volumes/volume-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '80fa90a7-dc37-41b3-9de3-7d7c37cc72e1', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'cad1458a-c458-4441-861f-904feaa91aec'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:33:16.338197 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-e3db5609-ad35-4c28-ace8-3b6541f6b176 req-0ec9ba36-a106-4ac1-94cf-a244d69c8b11 tempest-AttachVolumeNegativeTest-1091756803 None] attachment_update completed successfully. Sep 27 10:33:19.815207 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 req-ab1f4974-61c0-4255-899d-a8fcb23e847f tempest-AttachVolumeNegativeTest-1091756803 None] Terminate volume connection completed successfully. Sep 27 10:33:19.815916 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 req-ab1f4974-61c0-4255-899d-a8fcb23e847f tempest-AttachVolumeNegativeTest-1091756803 None] Deleting attachment cad1458a-c458-4441-861f-904feaa91aec. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:33:25.854351 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] Acquiring lock "cinder-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:33:25.887871 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] Lock "cinder-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1-delete_volume" acquired by "delete_volume" :: waited 0.032s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:33:25.958324 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:26.045598 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:33:26.046028 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] Volume volume-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:33:26.052919 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] deleting rbd volume volume-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:33:26.280545 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] Created reservations ['3f48a6ea-6e03-41fe-acd6-70b33c7fa3f0', 'ba16e106-7c59-4793-9399-aaaf6e3aad11', '13b194ac-4670-4f4f-b478-05c88d00bc15', '3367d954-2d36-41ef-bd97-1a6b737fbda1'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:33:26.326527 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:26.356152 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:26.381767 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:33:26.386765 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] Deleted volume successfully. Sep 27 10:33:26.400018 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b0760842-198b-422b-80e2-78ce70ed0664 tempest-AttachVolumeNegativeTest-1091756803 None] Lock "cinder-80fa90a7-dc37-41b3-9de3-7d7c37cc72e1-delete_volume" released by "delete_volume" :: held 0.514s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:33:27.109095 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] Acquiring lock "cinder-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:33:27.130708 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] Lock "cinder-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:33:27.170062 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:27.210025 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:33:27.210440 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] Volume volume-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:33:27.214943 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] deleting rbd volume volume-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:33:27.460917 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] Created reservations ['89aa5786-4f51-4af0-a237-7d67165b4e3f', 'ff5e36e3-3dd2-4661-aa5d-03baacfea4dd', '432e1248-12d9-4ef7-9bf3-bb100564ea3b', '033c8b78-2676-495a-89cb-abcdb3f639c3'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:33:27.507771 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:27.536969 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:27.567172 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:33:27.571745 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] Deleted volume successfully. Sep 27 10:33:27.586855 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-38acfd8a-52b1-4ca3-b1f0-f1063ec8fe5e tempest-AttachVolumeNegativeTest-1091756803 None] Lock "cinder-4f4a8bff-8624-4fb4-b665-2f8273aaf6ea-delete_volume" released by "delete_volume" :: held 0.456s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:33:28.041109 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 req-b1b9fe18-e2ff-4e2c-a8c4-0a432146ff3f tempest-TestVolumeBootPattern-4600220 None] Terminate volume connection completed successfully. Sep 27 10:33:28.041671 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 req-b1b9fe18-e2ff-4e2c-a8c4-0a432146ff3f tempest-TestVolumeBootPattern-4600220 None] Deleting attachment 50e881dc-ef77-42cc-8ab3-8591492443d4. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:33:28.472264 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] Acquiring lock "cinder-64e8066b-82ca-4278-ac3b-4ce6d3460710-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:33:28.494022 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] Lock "cinder-64e8066b-82ca-4278-ac3b-4ce6d3460710-delete_volume" acquired by "delete_volume" :: waited 0.025s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:33:28.544219 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:28.614204 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:33:28.614638 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] Volume volume-64e8066b-82ca-4278-ac3b-4ce6d3460710 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:33:28.620891 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] deleting rbd volume volume-64e8066b-82ca-4278-ac3b-4ce6d3460710 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:33:28.822161 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] Created reservations ['1f08cd12-cfe0-4caf-9c11-459f9ebd8a18', 'a20f4f66-08fb-4284-abe6-ad3049f746d1', '4cb1732d-bb1e-4eff-a7b4-c875f3055457', '636e77c8-e2d8-4271-9a25-b91219e35670'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:33:28.879076 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:28.904226 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:28.929266 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:33:28.933986 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] Deleted volume successfully. Sep 27 10:33:28.948685 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1b28a997-09ff-46e5-a7bb-b76f58282e3c tempest-AttachVolumeNegativeTest-1091756803 None] Lock "cinder-64e8066b-82ca-4278-ac3b-4ce6d3460710-delete_volume" released by "delete_volume" :: held 0.455s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:33:29.291323 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-603c151d-8628-4a8d-9bf1-18805cfe9928 req-4d30a5ea-0bb5-4a6e-9f04-2002f1daa1de tempest-ServerRescueNegativeTestJSON-1875151076 None] Terminate volume connection completed successfully. Sep 27 10:33:29.291733 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-603c151d-8628-4a8d-9bf1-18805cfe9928 req-4d30a5ea-0bb5-4a6e-9f04-2002f1daa1de tempest-ServerRescueNegativeTestJSON-1875151076 None] Deleting attachment 040d0e4e-a4ac-4258-a0c8-dce7cd95af51. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:33:29.687560 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c5ed40f7-dcea-4fa9-87e3-e2df1b6aa0d2 tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-c2c6c4b3-ba4e-4994-8674-d39521abb223-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:33:29.710322 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c5ed40f7-dcea-4fa9-87e3-e2df1b6aa0d2 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c2c6c4b3-ba4e-4994-8674-d39521abb223-delete_volume" acquired by "delete_volume" :: waited 0.023s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:33:29.735881 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=67,cinder:DELETE=1,cinder:UPDATE=28,cinder:INSERT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:29.766056 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c5ed40f7-dcea-4fa9-87e3-e2df1b6aa0d2 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:29.865152 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c5ed40f7-dcea-4fa9-87e3-e2df1b6aa0d2 tempest-TestVolumeBootPattern-4600220 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:33:29.873842 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c5ed40f7-dcea-4fa9-87e3-e2df1b6aa0d2 tempest-TestVolumeBootPattern-4600220 None] deleting rbd volume volume-c2c6c4b3-ba4e-4994-8674-d39521abb223 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:33:29.875750 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:33:29.876325 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:29.926034 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:29.981859 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:33:30.179838 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=60,cinder:UPDATE=29,cinder:INSERT=8,cinder:DELETE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:30.210484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-c5ed40f7-dcea-4fa9-87e3-e2df1b6aa0d2 tempest-TestVolumeBootPattern-4600220 None] Created reservations ['ccaaa0f8-4de9-4612-a35a-f8c8d3d154f8', '42dcd4fe-b884-43b8-8f36-18899df7b3b6', 'a13ddcfc-416d-4664-abe7-31ca07dd591b', '4cc2ae56-c3f5-4a0d-88e7-1bb1b2f93be2'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:33:30.258095 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c5ed40f7-dcea-4fa9-87e3-e2df1b6aa0d2 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:30.295062 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c5ed40f7-dcea-4fa9-87e3-e2df1b6aa0d2 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:30.329996 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-c5ed40f7-dcea-4fa9-87e3-e2df1b6aa0d2 tempest-TestVolumeBootPattern-4600220 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:33:30.334415 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c5ed40f7-dcea-4fa9-87e3-e2df1b6aa0d2 tempest-TestVolumeBootPattern-4600220 None] Deleted volume successfully. Sep 27 10:33:30.351724 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-c5ed40f7-dcea-4fa9-87e3-e2df1b6aa0d2 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-c2c6c4b3-ba4e-4994-8674-d39521abb223-delete_volume" released by "delete_volume" :: held 0.641s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:33:30.860608 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1178a92d-1e3c-41fd-a40d-e0dc29e98793 tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-257d6e25-918d-4ac2-b78c-03b14811a9f2-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:33:30.871128 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1178a92d-1e3c-41fd-a40d-e0dc29e98793 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-257d6e25-918d-4ac2-b78c-03b14811a9f2-delete_snapshot" acquired by "delete_snapshot" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:33:30.872952 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1178a92d-1e3c-41fd-a40d-e0dc29e98793 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:35.247786 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:33:35.267096 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Flow 'volume_create_manager' (d89235ee-ec3f-4df0-b062-dfa284ed320b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:33:35.273599 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (13c783bc-0696-420e-8de4-fe10bfd8924a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:35.289537 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (13c783bc-0696-420e-8de4-fe10bfd8924a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:33:35Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotsTestJSON-volume-2118323264',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b974c5cb-3e23-4eef-87cd-fc18b0b71dde,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='78c18800a5fc4f40994fade5d422694d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:33:35Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:33:35Z,use_quota=True,user_id='f827322737bf4518b0b262ae61797687',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:35.294814 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b8398db1-a638-4c41-b379-885728851c41) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:35.298852 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b8398db1-a638-4c41-b379-885728851c41) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:35.303746 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (333bb287-89d3-4712-8e5c-0fea194ec1cf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:35.307696 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (333bb287-89d3-4712-8e5c-0fea194ec1cf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b974c5cb-3e23-4eef-87cd-fc18b0b71dde', 'volume_name': 'volume-b974c5cb-3e23-4eef-87cd-fc18b0b71dde', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:35.312424 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d4af46f6-ea12-4c07-a723-7067e1021e2a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:35.316216 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d4af46f6-ea12-4c07-a723-7067e1021e2a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:35.320716 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (614b3a67-f7a4-4ceb-8c47-a43137f71333) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:35.321792 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b974c5cb-3e23-4eef-87cd-fc18b0b71dde', 'volume_size': 1} Sep 27 10:33:35.322195 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] creating volume 'volume-b974c5cb-3e23-4eef-87cd-fc18b0b71dde' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:33:35.323614 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:35.372436 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:33:35.372878 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:33:35.376505 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (614b3a67-f7a4-4ceb-8c47-a43137f71333) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b974c5cb-3e23-4eef-87cd-fc18b0b71dde', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:35.380953 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dcac2b62-55b8-405f-a510-5ffcecc60047) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:33:35.391589 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Volume volume-b974c5cb-3e23-4eef-87cd-fc18b0b71dde (b974c5cb-3e23-4eef-87cd-fc18b0b71dde): created successfully Sep 27 10:33:35.395479 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dcac2b62-55b8-405f-a510-5ffcecc60047) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:33:35.401170 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Flow 'volume_create_manager' (d89235ee-ec3f-4df0-b062-dfa284ed320b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:33:35.411552 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 req-b956ae5d-b858-4721-ad2d-545efd8774c4 tempest-VolumesSnapshotsTestJSON-1036287016 None] Created volume successfully. Sep 27 10:33:36.565334 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-36cfd086-aa63-4fc5-af33-b93de8753768 req-5cee65a5-59ef-46dc-b6ce-e3155ac9566c tempest-VolumesSnapshotsTestJSON-1036287016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:36.868355 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Acquiring lock "cinder-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:33:36.877039 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Lock "cinder-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:33:36.910263 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:36.949088 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:33:36.949594 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Volume volume-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:33:36.954297 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] deleting rbd volume volume-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:33:37.147101 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Created reservations ['a8390e11-8e2c-4df9-87ae-fd4b934b66a3', '59583a21-7dd1-4d19-ae3d-35e339fc4299', 'a0dd0e2f-889c-4be9-b02f-f46d3553c9b9', 'e6f4646d-e92b-43f8-9e3e-6ac9ebace46a'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:33:37.194514 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:37.217609 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:37.241027 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:33:37.245568 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Deleted volume successfully. Sep 27 10:33:37.259439 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8891fab6-cffa-441f-b794-743a6a7250d5 tempest-ServerRescueNegativeTestJSON-1875151076 None] Lock "cinder-a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3-delete_volume" released by "delete_volume" :: held 0.382s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:33:37.714820 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-36cfd086-aa63-4fc5-af33-b93de8753768 req-5cee65a5-59ef-46dc-b6ce-e3155ac9566c tempest-VolumesSnapshotsTestJSON-1036287016 None] Create snapshot completed successfully Sep 27 10:33:38.025611 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] Acquiring lock "cinder-c4b02a63-754a-4f44-ab9f-69541e872afc-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:33:38.035523 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] Lock "cinder-c4b02a63-754a-4f44-ab9f-69541e872afc-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:33:38.080056 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:38.137108 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:33:38.137691 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] Volume volume-c4b02a63-754a-4f44-ab9f-69541e872afc is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:33:38.142261 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] deleting rbd volume volume-c4b02a63-754a-4f44-ab9f-69541e872afc {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:33:38.343144 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] Created reservations ['abc1ab12-9840-405a-a18f-4a262b9dbe99', '254cb4d8-e131-4498-b06d-c88742e85f68', '7e53b8ac-d141-4913-beed-aface80cacbb', '3d3e0858-3d54-4eb3-a916-e7765af1285c'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:33:38.391771 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:38.421028 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:38.449104 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:33:38.453601 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] Deleted volume successfully. Sep 27 10:33:38.467362 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-51177fc7-d8e5-4657-8710-95821f20f1d8 tempest-ServerRescueNegativeTestJSON-1875151076 None] Lock "cinder-c4b02a63-754a-4f44-ab9f-69541e872afc-delete_volume" released by "delete_volume" :: held 0.432s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:33:38.818325 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-5eb0b9e9-0257-40c0-8e8e-116836cc01c6 req-ac2a19d9-33ce-4d7f-9247-84d889f6b170 tempest-VolumesSnapshotsTestJSON-1036287016 None] Acquiring lock "cinder-62092bd6-9a66-44d4-8fcb-d3d911809cdb-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:33:38.834739 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-5eb0b9e9-0257-40c0-8e8e-116836cc01c6 req-ac2a19d9-33ce-4d7f-9247-84d889f6b170 tempest-VolumesSnapshotsTestJSON-1036287016 None] Lock "cinder-62092bd6-9a66-44d4-8fcb-d3d911809cdb-delete_snapshot" acquired by "delete_snapshot" :: waited 0.016s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:33:38.836334 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-5eb0b9e9-0257-40c0-8e8e-116836cc01c6 req-ac2a19d9-33ce-4d7f-9247-84d889f6b170 tempest-VolumesSnapshotsTestJSON-1036287016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:39.761436 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [req-5eb0b9e9-0257-40c0-8e8e-116836cc01c6 req-ac2a19d9-33ce-4d7f-9247-84d889f6b170 tempest-VolumesSnapshotsTestJSON-1036287016 None] Created reservations ['1b0f8ff0-2100-426c-bb11-0f4c43bd633a', 'd18a211d-1873-43a7-905e-2b942137d1bd', '56719f09-6caa-4c47-837d-b2256accfc3b', '4cc428f5-0dcf-45b0-acbb-0524b17c3b51'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:33:39.804166 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-5eb0b9e9-0257-40c0-8e8e-116836cc01c6 req-ac2a19d9-33ce-4d7f-9247-84d889f6b170 tempest-VolumesSnapshotsTestJSON-1036287016 None] Delete snapshot completed successfully. Sep 27 10:33:39.809499 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-5eb0b9e9-0257-40c0-8e8e-116836cc01c6 req-ac2a19d9-33ce-4d7f-9247-84d889f6b170 tempest-VolumesSnapshotsTestJSON-1036287016 None] Lock "cinder-62092bd6-9a66-44d4-8fcb-d3d911809cdb-delete_snapshot" released by "delete_snapshot" :: held 0.975s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:33:39.989031 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] Acquiring lock "cinder-b974c5cb-3e23-4eef-87cd-fc18b0b71dde-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:33:39.997858 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] Lock "cinder-b974c5cb-3e23-4eef-87cd-fc18b0b71dde-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:33:40.008245 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=59,cinder:INSERT=11,cinder:UPDATE=27,cinder:DELETE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:40.035891 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:40.074157 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:33:40.074714 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] Volume volume-b974c5cb-3e23-4eef-87cd-fc18b0b71dde is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:33:40.081105 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] deleting rbd volume volume-b974c5cb-3e23-4eef-87cd-fc18b0b71dde {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:33:40.280002 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=55,cinder:UPDATE=38,cinder:INSERT=5,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:40.296116 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] Created reservations ['10b50b70-ed71-4528-b3cb-f2fd4e023ef7', '219fad0c-c16b-4eba-b95f-e26a0e70b0a0', 'b82e30cf-57ec-41c3-b576-ffd77cdb052b', 'c6ec5e35-c65c-4120-a8d3-ea59a7e45323'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:33:40.354840 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:40.378043 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:33:40.402054 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:33:40.406241 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] Deleted volume successfully. Sep 27 10:33:40.420898 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 req-e7033234-6705-4690-af64-1565305d6820 tempest-VolumesSnapshotsTestJSON-1036287016 None] Lock "cinder-b974c5cb-3e23-4eef-87cd-fc18b0b71dde-delete_volume" released by "delete_volume" :: held 0.423s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:34:01.096244 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-1178a92d-1e3c-41fd-a40d-e0dc29e98793 tempest-TestVolumeBootPattern-4600220 None] Created reservations ['40556b9e-c61f-4b36-892f-1823a16ebe07', '21e6414a-82b9-4e13-8687-c89c585853f6', '345dc474-ed0a-4632-81a3-623acb9235ae', '8db87e2b-00b4-44b2-aab7-1cb135bf74a9'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:34:01.158117 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1178a92d-1e3c-41fd-a40d-e0dc29e98793 tempest-TestVolumeBootPattern-4600220 None] Delete snapshot completed successfully. Sep 27 10:34:01.162965 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1178a92d-1e3c-41fd-a40d-e0dc29e98793 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-257d6e25-918d-4ac2-b78c-03b14811a9f2-delete_snapshot" released by "delete_snapshot" :: held 30.292s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:34:07.764559 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf req-276e57e7-30b4-4348-ab8a-f9d76959bfd9 tempest-TestVolumeBootPattern-4600220 None] Terminate volume connection completed successfully. Sep 27 10:34:07.765243 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf req-276e57e7-30b4-4348-ab8a-f9d76959bfd9 tempest-TestVolumeBootPattern-4600220 None] Deleting attachment 2700969a-6927-4e00-88af-2a1c3977077a. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:34:10.777476 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] Acquiring lock "cinder-8489f678-48b9-47ca-8f5e-397c453365e4-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:34:10.816752 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-8489f678-48b9-47ca-8f5e-397c453365e4-delete_volume" acquired by "delete_volume" :: waited 0.035s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:34:10.819628 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=26,cinder:INSERT=8,cinder:UPDATE=13,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:10.838022 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=16,cinder:SELECT=17 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:10.900807 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:34:11.063546 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:34:11.075641 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] deleting rbd volume volume-8489f678-48b9-47ca-8f5e-397c453365e4 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:34:11.645958 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:34:11.716161 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] deleting parent snapshot volume-8489f678-48b9-47ca-8f5e-397c453365e4.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:34:12.074538 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] Created reservations ['bc5b0e77-1380-433f-9310-f9479fc480b9', '19bcee99-c3bf-485e-bf3b-cc0068e0b551', 'cf253a66-b141-46ae-bdd3-f46116fbcb7c', 'ab9aa68c-fd8a-4938-be83-e566d50e52c1'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:34:12.161948 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:34:12.209271 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:34:12.273143 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:34:12.285101 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] Deleted volume successfully. Sep 27 10:34:12.309979 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-75c9152b-03a3-446e-b33c-3b631208f918 tempest-TestVolumeBootPattern-4600220 None] Lock "cinder-8489f678-48b9-47ca-8f5e-397c453365e4-delete_volume" released by "delete_volume" :: held 1.495s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:34:27.630464 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=12,cinder:UPDATE=5,cinder:INSERT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:29.874973 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:34:29.875853 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:34:29.941030 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:34:29.986280 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:34:47.629072 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=17,cinder:INSERT=2,cinder:UPDATE=13,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:47.647642 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:07.641553 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:27.635568 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:29.873086 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:35:29.873761 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:35:29.931778 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:35:29.994974 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:35:37.644034 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:57.642579 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:07.648226 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=10,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:17.658744 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:27.658735 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:29.876064 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:36:29.876872 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:36:29.964539 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:36:30.011269 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:36:47.654452 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:57.660307 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:59.815219 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-8607b4f4-bd13-4b30-90ec-329d7c70c524 tempest-BackendsCapabilitiesAdminTestsJSON-1882753405 None] Obtained capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.3.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:e8b2ca7b-8933-4391-a7b9-e76577b88317:cinder:volumes', 'backend_state': 'up', 'qos_support': True, 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=138473) get_capabilities /opt/stack/cinder/cinder/volume/manager.py:4776}} Sep 27 10:36:59.860224 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-bd6e1f3e-a436-41a0-a9ed-2471ba589da4 tempest-BackendsCapabilitiesAdminTestsJSON-1882753405 None] Obtained capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.3.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:e8b2ca7b-8933-4391-a7b9-e76577b88317:cinder:volumes', 'backend_state': 'up', 'qos_support': True, 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=138473) get_capabilities /opt/stack/cinder/cinder/volume/manager.py:4776}} Sep 27 10:37:16.835864 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:37:16.854231 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (ea9881b9-d0c7-4360-9fc0-bf28cd26f510) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:16.860270 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2d7d6cd1-a856-4f8d-b237-50b3a3b1ea1f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:16.876888 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2d7d6cd1-a856-4f8d-b237-50b3a3b1ea1f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:37:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1975071491',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a9ee92fd-d934-4130-a5be-f76f616903c9,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='c0b5a82c37db47a386f18c98b77044a0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:37:17Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:37:17Z,use_quota=True,user_id='bc86651b19cb4fa498b566c5d76eb4e9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:16.882563 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8a3a19c7-c691-4421-8a90-05d138530dae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:16.886537 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8a3a19c7-c691-4421-8a90-05d138530dae) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:16.891387 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (703fa740-4e40-44cf-bdbf-249bb775cf92) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:16.895271 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (703fa740-4e40-44cf-bdbf-249bb775cf92) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a9ee92fd-d934-4130-a5be-f76f616903c9', 'volume_name': 'volume-a9ee92fd-d934-4130-a5be-f76f616903c9', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:16.903939 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b70dc3b2-ea3a-4e71-a3e2-8461e3f1a22d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:16.903939 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b70dc3b2-ea3a-4e71-a3e2-8461e3f1a22d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:16.908529 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4c7220fd-1c56-451e-b429-374a25e6c0dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:16.909550 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Volume a9ee92fd-d934-4130-a5be-f76f616903c9: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a9ee92fd-d934-4130-a5be-f76f616903c9', 'volume_size': 1} Sep 27 10:37:16.909949 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] creating volume 'volume-a9ee92fd-d934-4130-a5be-f76f616903c9' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:37:16.910860 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:16.976264 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:37:16.976692 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:37:16.980534 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4c7220fd-1c56-451e-b429-374a25e6c0dd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a9ee92fd-d934-4130-a5be-f76f616903c9', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:16.985683 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fddc0c25-1ad7-4f5b-8490-74707e6e2f3b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:16.997375 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Volume volume-a9ee92fd-d934-4130-a5be-f76f616903c9 (a9ee92fd-d934-4130-a5be-f76f616903c9): created successfully Sep 27 10:37:17.002214 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fddc0c25-1ad7-4f5b-8490-74707e6e2f3b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:17.007887 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (ea9881b9-d0c7-4360-9fc0-bf28cd26f510) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:17.018503 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-433dc0af-9455-41ae-9b5e-34516560a274 tempest-VolumesGetTest-1802440771 None] Created volume successfully. Sep 27 10:37:18.462469 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=11,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:18.471678 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:37:18.492196 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (2fd48323-120e-4d08-b612-92bf8a8a0a2d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:18.498339 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7327fe25-3867-4598-bd5e-6e25ed0a88ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:18.514998 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7327fe25-3867-4598-bd5e-6e25ed0a88ba) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:37:18Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-311934524',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c0b5a82c37db47a386f18c98b77044a0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:37:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:37:18Z,use_quota=True,user_id='bc86651b19cb4fa498b566c5d76eb4e9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:18.520806 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c2f5c025-989c-4e41-9fc6-5664b2dafa89) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:18.524857 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c2f5c025-989c-4e41-9fc6-5664b2dafa89) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:18.529609 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (49140935-e9b2-4b09-8479-54c8ad8243e9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:18.533483 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (49140935-e9b2-4b09-8479-54c8ad8243e9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3', 'volume_name': 'volume-86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:18.539094 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f6d2b3cb-a068-4ea1-a1c3-cb0f6d769c49) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:18.543057 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f6d2b3cb-a068-4ea1-a1c3-cb0f6d769c49) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:18.547670 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8b75a84b-89a2-452d-80d5-b2e6f85f5775) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:18.548779 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Volume 86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3', 'volume_size': 1} Sep 27 10:37:18.549170 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] creating volume 'volume-86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:37:18.549852 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:18.614305 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:37:18.614849 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:37:18.618621 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8b75a84b-89a2-452d-80d5-b2e6f85f5775) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:18.622949 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c9c393f1-8eb5-4c92-90e1-205477f49414) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:18.632880 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Volume volume-86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3 (86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3): created successfully Sep 27 10:37:18.636529 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c9c393f1-8eb5-4c92-90e1-205477f49414) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:18.641949 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (2fd48323-120e-4d08-b612-92bf8a8a0a2d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:18.651508 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-9102ca8d-eb7e-48b1-9014-934d6d412e17 tempest-VolumesGetTest-1802440771 None] Created volume successfully. Sep 27 10:37:19.586994 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] Acquiring lock "cinder-86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:19.602522 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] Lock "cinder-86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:19.649310 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:19.688942 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:37:19.689501 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] Volume volume-86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:19.696641 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] deleting rbd volume volume-86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:37:19.894175 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] Created reservations ['60e48acf-0254-4450-bf38-d486d8ff1a70', '5acb5360-3b0c-4613-885b-18f39f77e573', '5cbccd9a-6e23-425f-9323-234461932079', '24ef0d82-0b48-4f35-967b-08ccf1a1f874'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:19.945580 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:19.985636 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:20.016110 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:20.020641 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] Deleted volume successfully. Sep 27 10:37:20.034310 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e2e833e6-4692-41b0-b7d6-7e205daa391d tempest-VolumesGetTest-1802440771 None] Lock "cinder-86e8dfe1-4360-41ab-bcad-3e1bb2aa05a3-delete_volume" released by "delete_volume" :: held 0.432s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:20.742080 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] Acquiring lock "cinder-a9ee92fd-d934-4130-a5be-f76f616903c9-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:20.759922 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] Lock "cinder-a9ee92fd-d934-4130-a5be-f76f616903c9-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:20.823583 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:20.863437 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:37:20.863860 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] Volume volume-a9ee92fd-d934-4130-a5be-f76f616903c9 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:20.869552 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] deleting rbd volume volume-a9ee92fd-d934-4130-a5be-f76f616903c9 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:37:21.086510 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] Created reservations ['0b58d006-dde5-4e1c-accd-a2a007e79b0a', '0afc4e85-e2d5-43fd-9797-94df7e0ef553', '8468d158-3110-4a16-98d9-9f35c7de6574', '5379aa00-c07d-4afb-ae00-754543c7ccb6'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:21.132030 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:21.170536 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:21.204241 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:21.211278 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] Deleted volume successfully. Sep 27 10:37:21.225457 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b44d8f57-b22d-4700-9ecc-0a3c308b86cc tempest-VolumesGetTest-1802440771 None] Lock "cinder-a9ee92fd-d934-4130-a5be-f76f616903c9-delete_volume" released by "delete_volume" :: held 0.465s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:22.196934 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:37:22.220883 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (c9ed1b78-5526-4548-8246-e97b66741758) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:22.227483 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (43f379a3-b6e2-4d03-894a-02e0e965ee62) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:22.243629 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (43f379a3-b6e2-4d03-894a-02e0e965ee62) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:37:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-84840181',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8a6baf29-31e2-4f0d-a3d5-d56d551a6c80,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c0b5a82c37db47a386f18c98b77044a0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:37:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:37:22Z,use_quota=True,user_id='bc86651b19cb4fa498b566c5d76eb4e9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:22.248647 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (225c1daa-adc2-444b-9edd-3c07bb5faed9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:22.252629 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (225c1daa-adc2-444b-9edd-3c07bb5faed9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:22.257982 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (800e8a20-c4b1-4068-a52f-0a38ddf66018) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:22.261895 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (800e8a20-c4b1-4068-a52f-0a38ddf66018) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8a6baf29-31e2-4f0d-a3d5-d56d551a6c80', 'volume_name': 'volume-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:22.266598 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (28e70ea5-0284-4830-83e8-1e39a249fd6c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:22.270584 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (28e70ea5-0284-4830-83e8-1e39a249fd6c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:22.274979 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e0d71cce-89fa-4b9b-9243-61a4468cca20) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:22.276022 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Volume 8a6baf29-31e2-4f0d-a3d5-d56d551a6c80: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80', 'volume_size': 1} Sep 27 10:37:22.276412 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] creating volume 'volume-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:37:22.277199 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:22.336067 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:37:22.336067 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:37:22.339913 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e0d71cce-89fa-4b9b-9243-61a4468cca20) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:22.344338 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f1a3537e-4d8a-4009-a0d9-357821d4cfae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:22.354499 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Volume volume-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80 (8a6baf29-31e2-4f0d-a3d5-d56d551a6c80): created successfully Sep 27 10:37:22.360147 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f1a3537e-4d8a-4009-a0d9-357821d4cfae) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:22.364023 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (c9ed1b78-5526-4548-8246-e97b66741758) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:22.373944 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-689b5c6e-2b62-4e4d-9d10-b612ca29d189 tempest-VolumesGetTest-1802440771 None] Created volume successfully. Sep 27 10:37:23.576066 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:37:23.603173 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (e561f9d8-5b55-4265-800f-4c17f48459dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:23.609715 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7a31eaf9-5b7b-4dae-829e-cb644189891d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:23.627766 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7a31eaf9-5b7b-4dae-829e-cb644189891d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:37:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1959815895',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=058c4f5e-ccd6-4a12-850a-2ed4f79fca7e,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='c0b5a82c37db47a386f18c98b77044a0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:37:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=8a6baf29-31e2-4f0d-a3d5-d56d551a6c80,status='creating',terminated_at=None,updated_at=2022-09-27T10:37:24Z,use_quota=True,user_id='bc86651b19cb4fa498b566c5d76eb4e9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:23.633092 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f93edcfe-9a63-4a63-8040-d89eb31e60f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:23.637073 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f93edcfe-9a63-4a63-8040-d89eb31e60f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:23.641930 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (41e1e28e-4cf5-496b-a1e4-fa1ef962a4a9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:23.657753 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (41e1e28e-4cf5-496b-a1e4-fa1ef962a4a9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '058c4f5e-ccd6-4a12-850a-2ed4f79fca7e', 'volume_name': 'volume-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e', 'volume_size': 1, 'source_volid': '8a6baf29-31e2-4f0d-a3d5-d56d551a6c80', 'source_volstatus': 'available'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:23.662729 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (57daec18-3f38-46da-a08d-64368bad819a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:23.666783 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (57daec18-3f38-46da-a08d-64368bad819a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:23.671547 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (68397074-9378-48da-9b0f-1e03b05f5fa8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:23.672643 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Volume 058c4f5e-ccd6-4a12-850a-2ed4f79fca7e: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e', 'volume_size': 1, 'source_volid': '8a6baf29-31e2-4f0d-a3d5-d56d551a6c80', 'source_volstatus': 'available'} Sep 27 10:37:23.686183 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:23.731292 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] creating snapshot='volume-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:37:24.365880 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] cloning 'volume-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80@volume-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e.clone_snap' to 'volume-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:37:24.451670 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Volume volume-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:24.452887 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:37:24.453258 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:37:24.471017 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:37:24.475269 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (68397074-9378-48da-9b0f-1e03b05f5fa8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e', 'volume_size': 1, 'source_volid': '8a6baf29-31e2-4f0d-a3d5-d56d551a6c80', 'source_volstatus': 'available'}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:24.479711 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8fcbaca4-1b4f-403e-8b21-4490c458b376) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:24.490269 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Volume volume-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e (058c4f5e-ccd6-4a12-850a-2ed4f79fca7e): created successfully Sep 27 10:37:24.493967 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8fcbaca4-1b4f-403e-8b21-4490c458b376) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:24.499288 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (e561f9d8-5b55-4265-800f-4c17f48459dd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:24.515246 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5ecf4a03-95b8-4daa-9dda-be247d694e2e tempest-VolumesGetTest-1802440771 None] Created volume successfully. Sep 27 10:37:25.162816 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:37:25.182957 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (1df28076-4403-480e-b83a-0dc0f822da02) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:25.189420 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (98ea4216-6c8d-4000-9a35-1443f04bea25) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:25.206501 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (98ea4216-6c8d-4000-9a35-1443f04bea25) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:37:25Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-2077247495',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c0b5a82c37db47a386f18c98b77044a0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:37:25Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:37:25Z,use_quota=True,user_id='bc86651b19cb4fa498b566c5d76eb4e9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:25.211946 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (babce17a-5074-4bdc-8307-d22a2b627a5b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:25.216143 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (babce17a-5074-4bdc-8307-d22a2b627a5b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:25.221175 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e9fa9739-6a7a-4a29-bd9d-2eec860f2c81) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:25.225151 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e9fa9739-6a7a-4a29-bd9d-2eec860f2c81) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1', 'volume_name': 'volume-41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:25.229840 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (409a1ebc-4a5f-47e7-94b8-535f1f925ca6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:25.233624 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (409a1ebc-4a5f-47e7-94b8-535f1f925ca6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:25.238159 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4fb02b13-b174-4c9c-a943-054e57c49fac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:25.239340 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Volume 41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1', 'volume_size': 1} Sep 27 10:37:25.239610 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] creating volume 'volume-41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:37:25.240612 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:25.302403 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:37:25.302862 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:37:25.306699 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4fb02b13-b174-4c9c-a943-054e57c49fac) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:25.311077 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (771a477f-f481-468a-8b1b-eb10b922c8be) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:25.321625 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Volume volume-41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1 (41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1): created successfully Sep 27 10:37:25.325564 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (771a477f-f481-468a-8b1b-eb10b922c8be) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:25.330849 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (1df28076-4403-480e-b83a-0dc0f822da02) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:25.346118 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d431bbba-4d09-4456-81a2-0ee5ed20297f tempest-VolumesGetTest-1802440771 None] Created volume successfully. Sep 27 10:37:26.277253 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] Acquiring lock "cinder-41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:26.287781 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] Lock "cinder-41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:26.325587 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:26.375720 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:37:26.376149 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] Volume volume-41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:26.381896 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] deleting rbd volume volume-41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:37:26.549300 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=61,cinder:UPDATE=29,cinder:DELETE=6,cinder:INSERT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:26.567769 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] Created reservations ['b6bdb137-9540-4b7c-b17e-697045a1c1f1', '29f509da-a1ee-4b1b-9676-683ee6e77d6f', 'cbf65cb9-617b-4bcc-aa6d-f8a7236ea596', 'fe086d04-3ae1-436c-aea2-cf47e65c9eff'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:26.616392 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:26.642589 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:26.669524 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:26.673883 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] Deleted volume successfully. Sep 27 10:37:26.691058 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-611cf906-3b89-492b-be58-04cf57eb6ae0 tempest-VolumesGetTest-1802440771 None] Lock "cinder-41ae02c4-9fa8-4ee4-9e5a-e392dba13ff1-delete_volume" released by "delete_volume" :: held 0.403s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:27.418609 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] Acquiring lock "cinder-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:27.428074 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] Lock "cinder-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:27.467907 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:27.521065 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:37:27.529289 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] deleting rbd volume volume-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:37:27.690574 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:37:27.709308 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] Volume volume-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:27.710007 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] deleting parent snapshot volume-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:37:28.457813 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=62,cinder:INSERT=8,cinder:UPDATE=29,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:28.471269 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] Created reservations ['edbcef0c-a0bb-4e1f-a293-e499b7882312', '6484f20a-0e73-49b8-bb53-cf41d96ee9d9', '852697a1-9487-42ec-831a-c749e149d458', 'b4889dfe-49ee-4422-bce8-69db1f62527b'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:28.529599 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:28.606528 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:28.640997 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:28.648015 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] Deleted volume successfully. Sep 27 10:37:28.659105 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-79398e9a-b858-4066-9e18-d80fab5a135f tempest-GroupSnapshotsV319Test-1853126775 None] Group tempest-GroupSnapshotsV319Test-Group-969363785: creating Sep 27 10:37:28.670446 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3c75048f-b6ae-4ff5-896b-10d1b23da062 tempest-VolumesGetTest-1802440771 None] Lock "cinder-058c4f5e-ccd6-4a12-850a-2ed4f79fca7e-delete_volume" released by "delete_volume" :: held 1.243s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:28.682156 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-79398e9a-b858-4066-9e18-d80fab5a135f tempest-GroupSnapshotsV319Test-1853126775 None] Group tempest-GroupSnapshotsV319Test-Group-969363785: created successfully Sep 27 10:37:28.691948 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-79398e9a-b858-4066-9e18-d80fab5a135f tempest-GroupSnapshotsV319Test-1853126775 None] Create group completed successfully. Sep 27 10:37:29.033881 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:37:29.052425 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (fec1a483-bc46-4c84-89f9-de4d6c86b222) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:29.058472 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b4a95cf3-b288-4f19-abd6-95a4750eb4c4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:29.074479 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b4a95cf3-b288-4f19-abd6-95a4750eb4c4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:37:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1260765756',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2a107201-ae80-4ce8-8542-60678af4b84a,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='c0b5a82c37db47a386f18c98b77044a0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:37:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:37:29Z,use_quota=True,user_id='bc86651b19cb4fa498b566c5d76eb4e9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:29.079526 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2b9934b6-e7d0-4cf6-9f7e-4fa6c6f5cb07) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:29.083655 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2b9934b6-e7d0-4cf6-9f7e-4fa6c6f5cb07) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:29.088430 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f7dc10a6-7eb1-4edd-a51d-0e4ce2d239bc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:29.171473 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f7dc10a6-7eb1-4edd-a51d-0e4ce2d239bc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '2a107201-ae80-4ce8-8542-60678af4b84a', 'volume_name': 'volume-2a107201-ae80-4ce8-8542-60678af4b84a', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:29.176273 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (90f58efb-cb8a-4e11-a369-28441e90c945) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:29.180069 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (90f58efb-cb8a-4e11-a369-28441e90c945) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:29.184526 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b480c73a-3be8-4b49-9581-41e898e344ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:29.185654 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Volume 2a107201-ae80-4ce8-8542-60678af4b84a: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-2a107201-ae80-4ce8-8542-60678af4b84a', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:37:29.186203 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Cloning 2a107201-ae80-4ce8-8542-60678af4b84a from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:37:29.186954 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:29.195727 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:29.207896 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 37, 29, 200349)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:37:29.208343 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:37:29.208825 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:37:29.213087 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.017s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:29.214041 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:37:29.219815 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 37, 29, 217503)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:37:29.220239 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:37:29.220608 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:37:29.232365 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:29.284866 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] creating snapshot='volume-2a107201-ae80-4ce8-8542-60678af4b84a.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:37:30.016749 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-2a107201-ae80-4ce8-8542-60678af4b84a.clone_snap' to 'volume-2a107201-ae80-4ce8-8542-60678af4b84a' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:37:30.023460 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:37:30.024548 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:30.066040 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:30.093976 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:30.114380 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:37:30.134179 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Flow 'volume_create_manager' (f2c38b47-d96c-4798-ac42-ddd09c5fda52) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:30.140407 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3c00c8ce-9ea2-4b6b-aaa7-4d7b9b11b696) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:30.180099 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:30.181429 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:37:30.181807 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:37:30.205352 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:37:30.205995 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Marking volume 2a107201-ae80-4ce8-8542-60678af4b84a as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:37:30.222993 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3c00c8ce-9ea2-4b6b-aaa7-4d7b9b11b696) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:37:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsV319Test-Volume-1610129133',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=9b453fcc-447a-4108-adf5-643ee19f5dc3,host='ncdv-master-1@ceph#ceph',id=e84aef77-2879-4d48-bea8-65e0d2722b1e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='083f8a9f05154994b0cd6423c431cc0d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:37:30Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:37:30Z,use_quota=True,user_id='8e454d008f3b43088c2dbc2e6ac0e4e4',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(08b05b2b-00c9-4233-8c78-2f053ed87d6a),volume_type_id=08b05b2b-00c9-4233-8c78-2f053ed87d6a)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:30.228146 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (55234837-2f39-4c83-ae6c-5d9b1168bdfb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:30.232195 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (55234837-2f39-4c83-ae6c-5d9b1168bdfb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:30.237089 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (680e2cfc-00e0-4031-9e75-83ee583a64e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:30.241173 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (680e2cfc-00e0-4031-9e75-83ee583a64e1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e84aef77-2879-4d48-bea8-65e0d2722b1e', 'volume_name': 'volume-e84aef77-2879-4d48-bea8-65e0d2722b1e', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:30.245847 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (548e8f38-a631-4ebc-890e-4d41203ad859) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:30.249599 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (548e8f38-a631-4ebc-890e-4d41203ad859) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:30.254270 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (99ae88ca-dbdf-45fa-be8b-300062833862) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:30.255544 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Volume e84aef77-2879-4d48-bea8-65e0d2722b1e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e84aef77-2879-4d48-bea8-65e0d2722b1e', 'volume_size': 1} Sep 27 10:37:30.255755 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] creating volume 'volume-e84aef77-2879-4d48-bea8-65e0d2722b1e' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:37:30.256516 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:30.278462 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 2a107201-ae80-4ce8-8542-60678af4b84a. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:37:30.278941 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Creating volume glance metadata for volume 2a107201-ae80-4ce8-8542-60678af4b84a backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:37:30.315985 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] extra_specs: {} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:37:30.316457 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] extra_specs: {} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:37:30.320546 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (99ae88ca-dbdf-45fa-be8b-300062833862) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e84aef77-2879-4d48-bea8-65e0d2722b1e', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:30.325162 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b16b0a28-733f-44b4-acd3-367289378afc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:30.337673 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Volume volume-e84aef77-2879-4d48-bea8-65e0d2722b1e (e84aef77-2879-4d48-bea8-65e0d2722b1e): created successfully Sep 27 10:37:30.341134 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b16b0a28-733f-44b4-acd3-367289378afc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:30.346508 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Flow 'volume_create_manager' (f2c38b47-d96c-4798-ac42-ddd09c5fda52) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:30.359858 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-aeb12b1c-a7cf-40fc-b030-7be7f6b08963 tempest-GroupSnapshotsV319Test-1853126775 None] Created volume successfully. Sep 27 10:37:30.375271 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b480c73a-3be8-4b49-9581-41e898e344ca) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2a107201-ae80-4ce8-8542-60678af4b84a', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:30.379969 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0aa203ba-8d18-4302-80cd-8893b4e7ceb1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:30.389510 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Volume volume-2a107201-ae80-4ce8-8542-60678af4b84a (2a107201-ae80-4ce8-8542-60678af4b84a): created successfully Sep 27 10:37:30.393216 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0aa203ba-8d18-4302-80cd-8893b4e7ceb1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:30.398574 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (fec1a483-bc46-4c84-89f9-de4d6c86b222) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:30.408089 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-edde6b17-8fbb-483c-af59-f8635d9f96e5 tempest-VolumesGetTest-1802440771 None] Created volume successfully. Sep 27 10:37:31.289689 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7bf48837-8a26-41dc-bedd-e1afdf9ef44c tempest-GroupSnapshotsV319Test-1853126775 None] GroupSnapshot 0fe9b03e-d5c9-49a7-be13-08270eb32d74: creating. Sep 27 10:37:31.312560 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-7bf48837-8a26-41dc-bedd-e1afdf9ef44c tempest-GroupSnapshotsV319Test-1853126775 None] Group snapshot 0fe9b03e-d5c9-49a7-be13-08270eb32d74: creating. {{(pid=138473) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Sep 27 10:37:31.331519 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7bf48837-8a26-41dc-bedd-e1afdf9ef44c tempest-GroupSnapshotsV319Test-1853126775 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:31.686784 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:37:31.705313 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (66e6e378-f33b-41e7-b846-027ba59b422d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:31.711512 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (899769ca-1c67-4217-b71c-655833ff9071) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:31.730411 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (899769ca-1c67-4217-b71c-655833ff9071) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:37:31Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-1724560733',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c0b5a82c37db47a386f18c98b77044a0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:37:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:37:32Z,use_quota=True,user_id='bc86651b19cb4fa498b566c5d76eb4e9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:31.735490 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (894252fd-bf34-492c-879d-ac88722345e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:31.739422 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (894252fd-bf34-492c-879d-ac88722345e5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:31.744135 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1b7617a5-e624-456d-a546-aa7c0e0efcca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:31.747934 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1b7617a5-e624-456d-a546-aa7c0e0efcca) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6', 'volume_name': 'volume-b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:31.752515 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6fd463fa-5e8f-4f25-aebd-6b4a38c70c7b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:31.756253 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6fd463fa-5e8f-4f25-aebd-6b4a38c70c7b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:31.760589 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (da1de674-4aea-4666-9d9e-997ca409eb57) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:31.761629 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Volume b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6', 'volume_size': 1} Sep 27 10:37:31.762003 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] creating volume 'volume-b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:37:31.762632 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:31.821655 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:37:31.822035 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:37:31.825634 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (da1de674-4aea-4666-9d9e-997ca409eb57) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:31.829968 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (780e0933-5707-4082-81c0-b3a646b1bc76) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:31.842265 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Volume volume-b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6 (b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6): created successfully Sep 27 10:37:31.845810 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (780e0933-5707-4082-81c0-b3a646b1bc76) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:31.851549 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Flow 'volume_create_manager' (66e6e378-f33b-41e7-b846-027ba59b422d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:31.861812 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-f4443d13-65a0-434e-91b5-4bcdfdbf1971 tempest-VolumesGetTest-1802440771 None] Created volume successfully. Sep 27 10:37:32.088928 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7bf48837-8a26-41dc-bedd-e1afdf9ef44c tempest-GroupSnapshotsV319Test-1853126775 None] group_snapshot 0fe9b03e-d5c9-49a7-be13-08270eb32d74: created successfully Sep 27 10:37:32.730650 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-15f656e1-8072-42f0-abcb-f846eea219fc tempest-GroupSnapshotsV319Test-1853126775 None] group_snapshot 0fe9b03e-d5c9-49a7-be13-08270eb32d74: deleting Sep 27 10:37:32.762020 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-15f656e1-8072-42f0-abcb-f846eea219fc tempest-GroupSnapshotsV319Test-1853126775 None] group_snapshot 0fe9b03e-d5c9-49a7-be13-08270eb32d74: deleting {{(pid=138473) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Sep 27 10:37:32.779563 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-15f656e1-8072-42f0-abcb-f846eea219fc tempest-GroupSnapshotsV319Test-1853126775 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:32.837923 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] Acquiring lock "cinder-b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:32.849183 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] Lock "cinder-b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:32.899812 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=27,cinder:SELECT=61,cinder:DELETE=5,cinder:INSERT=7 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:32.905640 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:32.945176 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:37:32.945615 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] Volume volume-b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:32.950174 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] deleting rbd volume volume-b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:37:33.082171 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-15f656e1-8072-42f0-abcb-f846eea219fc tempest-GroupSnapshotsV319Test-1853126775 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsV319Test-volume-type-773621514 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsV319Test-volume-type-773621514, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:37:33.082960 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-15f656e1-8072-42f0-abcb-f846eea219fc tempest-GroupSnapshotsV319Test-1853126775 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsV319Test-volume-type-773621514 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsV319Test-volume-type-773621514, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:37:33.101197 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-15f656e1-8072-42f0-abcb-f846eea219fc tempest-GroupSnapshotsV319Test-1853126775 None] Created reservations ['b7f8b20b-f48d-45aa-b351-5c8888a8099a', '04d5ed92-a03f-46e0-8167-7a13f229b046', '9d07e6e9-0e85-4b81-915d-9c7ea2113804', '6a231571-bc8f-4810-8cdb-0327636f3fdc'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:33.158345 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=23,cinder:SELECT=66,cinder:INSERT=11 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:33.167729 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-15f656e1-8072-42f0-abcb-f846eea219fc tempest-GroupSnapshotsV319Test-1853126775 None] group_snapshot 0fe9b03e-d5c9-49a7-be13-08270eb32d74: deleted successfully Sep 27 10:37:33.177159 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] Created reservations ['2a2f9fb6-5bf7-42db-9132-b87916291a3b', '8afaab52-8e4f-49ec-b79e-513daef25f69', 'd8e1ff69-f584-4cd9-9311-21dcbfe5f8af', '432a8078-495e-4bb0-aa96-1cbb751dadc2'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:33.227484 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:33.253941 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:33.281433 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:33.285834 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] Deleted volume successfully. Sep 27 10:37:33.299770 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d7942d09-db78-4322-9896-a471bce5eee5 tempest-VolumesGetTest-1802440771 None] Lock "cinder-b02349e1-da7d-4a37-a8b9-1d3aa1e7c9d6-delete_volume" released by "delete_volume" :: held 0.451s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:34.002841 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] Acquiring lock "cinder-2a107201-ae80-4ce8-8542-60678af4b84a-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:34.018899 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] Lock "cinder-2a107201-ae80-4ce8-8542-60678af4b84a-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:34.062417 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:34.129409 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:37:34.137312 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] deleting rbd volume volume-2a107201-ae80-4ce8-8542-60678af4b84a {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:37:34.209581 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f67528e-2ae8-4f7e-b484-a5e72ad53c6e tempest-GroupSnapshotsV319Test-1853126775 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:34.254056 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f67528e-2ae8-4f7e-b484-a5e72ad53c6e tempest-GroupSnapshotsV319Test-1853126775 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:37:34.254678 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f67528e-2ae8-4f7e-b484-a5e72ad53c6e tempest-GroupSnapshotsV319Test-1853126775 None] Volume volume-e84aef77-2879-4d48-bea8-65e0d2722b1e is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:34.263221 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f67528e-2ae8-4f7e-b484-a5e72ad53c6e tempest-GroupSnapshotsV319Test-1853126775 None] deleting rbd volume volume-e84aef77-2879-4d48-bea8-65e0d2722b1e {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:37:34.317566 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:37:34.348103 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] deleting parent snapshot volume-2a107201-ae80-4ce8-8542-60678af4b84a.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:37:35.131740 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] Created reservations ['31089629-4ec6-4074-862b-4231ec95391b', '164bf95f-9579-4ce6-8b52-5a9e856a3757', '0ef972bc-5e48-4be0-bb4c-acb287796aaa', '0818a120-199c-4660-be3b-4a094e2ff05b'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:35.136013 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-9f67528e-2ae8-4f7e-b484-a5e72ad53c6e tempest-GroupSnapshotsV319Test-1853126775 None] Created reservations ['418f9c05-2f04-4a78-aafd-719dada7d7a8'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:35.171373 ncdv-master-1 cinder-volume[138473]: WARNING cinder.quota [None req-9f67528e-2ae8-4f7e-b484-a5e72ad53c6e tempest-GroupSnapshotsV319Test-1853126775 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsV319Test-volume-type-773621514 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsV319Test-volume-type-773621514, it is now deprecated. Please use the default quota class for default quota. Sep 27 10:37:35.191054 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-9f67528e-2ae8-4f7e-b484-a5e72ad53c6e tempest-GroupSnapshotsV319Test-1853126775 None] Created reservations ['682222cf-2689-49e6-b6b6-25fa3114d7b3', 'ebdd0c02-7225-467a-957e-2d9d86663200', '6be97cc2-e214-4f50-8ce9-5d7d1ef8d428', '66f7b67a-3121-48ff-b198-1c34c3509f8c'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:35.196316 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:35.232475 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:35.259586 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:35.280189 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] Deleted volume successfully. Sep 27 10:37:35.298903 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-683d8497-9f08-4b31-9cef-8aafaace7507 tempest-VolumesGetTest-1802440771 None] Lock "cinder-2a107201-ae80-4ce8-8542-60678af4b84a-delete_volume" released by "delete_volume" :: held 1.280s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:35.358827 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f67528e-2ae8-4f7e-b484-a5e72ad53c6e tempest-GroupSnapshotsV319Test-1853126775 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:35.382480 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f67528e-2ae8-4f7e-b484-a5e72ad53c6e tempest-GroupSnapshotsV319Test-1853126775 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:35.406119 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-9f67528e-2ae8-4f7e-b484-a5e72ad53c6e tempest-GroupSnapshotsV319Test-1853126775 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:35.410004 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-9f67528e-2ae8-4f7e-b484-a5e72ad53c6e tempest-GroupSnapshotsV319Test-1853126775 None] Delete group completed successfully. Sep 27 10:37:36.217568 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] Acquiring lock "cinder-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:36.228712 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] Lock "cinder-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:36.271548 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:36.309022 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:37:36.309463 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] Volume volume-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:36.317668 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] deleting rbd volume volume-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:37:36.495679 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=59,cinder:INSERT=9,cinder:UPDATE=30,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:36.524441 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] Created reservations ['7fa6b8da-a4f9-4568-927c-518c8ae7d5f3', '2a7019a4-6222-4a90-8955-f510b65aee03', '2ca82a43-396b-4451-87c3-ad1bfcc728c7', '00569f89-1d3c-4f8f-8c8b-47c71584bbe2'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:36.571050 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:36.596922 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:36.628267 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:36.632515 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] Deleted volume successfully. Sep 27 10:37:36.647101 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b530b5e6-2f8d-49ca-9043-c498f249ae25 tempest-VolumesGetTest-1802440771 None] Lock "cinder-8a6baf29-31e2-4f0d-a3d5-d56d551a6c80-delete_volume" released by "delete_volume" :: held 0.418s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:39.309051 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:37:39.328320 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Flow 'volume_create_manager' (f783176f-d0d2-43ae-9a18-4f042fc3979c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:39.334728 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ba0862c0-2f35-4800-add2-63773cfdf247) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:39.350720 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ba0862c0-2f35-4800-add2-63773cfdf247) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:37:39Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotListTestJSON-Volume-798436911',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6a3f4155-55b2-4aae-8e1f-96102d5111fc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a63c9d35dcc44ddca8ac9e16673649dd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:37:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:37:39Z,use_quota=True,user_id='eeb17d2ed61f4c1b9eec83b63b0bd67e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:39.355937 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e70351df-03bf-4dbc-b94c-821e9b601c0a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:39.359930 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e70351df-03bf-4dbc-b94c-821e9b601c0a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:39.364990 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (66667965-b7a4-4918-8584-ca4d0c6fbb73) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:39.368905 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (66667965-b7a4-4918-8584-ca4d0c6fbb73) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6a3f4155-55b2-4aae-8e1f-96102d5111fc', 'volume_name': 'volume-6a3f4155-55b2-4aae-8e1f-96102d5111fc', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:39.373549 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2a7d7acf-86a6-4b45-a6a1-44acd1c7978d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:39.377340 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2a7d7acf-86a6-4b45-a6a1-44acd1c7978d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:39.381925 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ee8882fe-ea47-4164-9cf7-c217a7a52754) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:39.382998 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Volume 6a3f4155-55b2-4aae-8e1f-96102d5111fc: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6a3f4155-55b2-4aae-8e1f-96102d5111fc', 'volume_size': 1} Sep 27 10:37:39.383405 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] creating volume 'volume-6a3f4155-55b2-4aae-8e1f-96102d5111fc' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:37:39.384165 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:39.443930 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:37:39.444316 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:37:39.447938 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ee8882fe-ea47-4164-9cf7-c217a7a52754) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6a3f4155-55b2-4aae-8e1f-96102d5111fc', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:39.452366 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (da230eb1-6148-4c4e-add0-976cb506255e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:39.462689 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Volume volume-6a3f4155-55b2-4aae-8e1f-96102d5111fc (6a3f4155-55b2-4aae-8e1f-96102d5111fc): created successfully Sep 27 10:37:39.466232 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (da230eb1-6148-4c4e-add0-976cb506255e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:39.471790 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Flow 'volume_create_manager' (f783176f-d0d2-43ae-9a18-4f042fc3979c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:39.482465 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-296bb249-7e8f-40ef-905b-93115cf5b92c tempest-VolumesSnapshotListTestJSON-276023532 None] Created volume successfully. Sep 27 10:37:40.531933 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=40,cinder:SELECT=48,cinder:INSERT=10,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:40.549201 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5f736b86-e3e9-4852-ba2f-f235834d190a tempest-VolumesSnapshotListTestJSON-276023532 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:41.385513 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5f736b86-e3e9-4852-ba2f-f235834d190a tempest-VolumesSnapshotListTestJSON-276023532 None] Create snapshot completed successfully Sep 27 10:37:41.757027 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7f2cf627-7bac-4a47-a524-cac856f83197 tempest-VolumesSnapshotListTestJSON-276023532 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:42.397971 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7f2cf627-7bac-4a47-a524-cac856f83197 tempest-VolumesSnapshotListTestJSON-276023532 None] Create snapshot completed successfully Sep 27 10:37:42.964888 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0c4d3c07-9f10-46f0-b010-5ce4cf2c6e52 tempest-VolumesSnapshotListTestJSON-276023532 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:43.451628 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0c4d3c07-9f10-46f0-b010-5ce4cf2c6e52 tempest-VolumesSnapshotListTestJSON-276023532 None] Create snapshot completed successfully Sep 27 10:37:43.460614 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:37:43.479665 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Flow 'volume_create_manager' (f843fc00-1c58-4ca1-890e-cf628e446455) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:43.485764 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9d506b03-57c6-422e-b0d9-994e72a82492) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:43.502366 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9d506b03-57c6-422e-b0d9-994e72a82492) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:37:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeSnapshotQuotasNegativeTestJSON-Volume-1106726455',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b8e29dbc-206d-4092-ac49-441ace7c9012,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='7a6eaa5db43e4b1fb9a22a0f856d955d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:37:43Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:37:43Z,use_quota=True,user_id='ff151cf6ba214570a63235c66c5c6625',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:43.507798 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4b6a8fd1-c7e6-4b6b-b8f0-f1bb00a5f467) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:43.511709 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4b6a8fd1-c7e6-4b6b-b8f0-f1bb00a5f467) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:43.516511 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (353abe4d-8d0d-47b8-9a94-3a7a3f5c7da0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:43.520403 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (353abe4d-8d0d-47b8-9a94-3a7a3f5c7da0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b8e29dbc-206d-4092-ac49-441ace7c9012', 'volume_name': 'volume-b8e29dbc-206d-4092-ac49-441ace7c9012', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:43.525033 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a107b3e1-38df-4ece-aa4e-cced23c200b8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:43.528704 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a107b3e1-38df-4ece-aa4e-cced23c200b8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:43.533115 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bf8f14f3-f211-4136-bcec-cd3c53220ec8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:43.534133 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Volume b8e29dbc-206d-4092-ac49-441ace7c9012: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b8e29dbc-206d-4092-ac49-441ace7c9012', 'volume_size': 1} Sep 27 10:37:43.534524 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] creating volume 'volume-b8e29dbc-206d-4092-ac49-441ace7c9012' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:37:43.535147 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:43.594682 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:37:43.595059 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:37:43.599406 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bf8f14f3-f211-4136-bcec-cd3c53220ec8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b8e29dbc-206d-4092-ac49-441ace7c9012', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:43.603943 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f8334899-ddf0-41f9-9a83-69dcbd65ea1c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:43.615152 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Volume volume-b8e29dbc-206d-4092-ac49-441ace7c9012 (b8e29dbc-206d-4092-ac49-441ace7c9012): created successfully Sep 27 10:37:43.618645 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f8334899-ddf0-41f9-9a83-69dcbd65ea1c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:43.623939 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Flow 'volume_create_manager' (f843fc00-1c58-4ca1-890e-cf628e446455) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:43.634643 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b0cca6d1-ec88-4602-9421-6cb2852ace44 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Created volume successfully. Sep 27 10:37:44.666077 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-ff723f48-a3bc-4a4e-b88b-54238aeef968 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:44.711967 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b04dd203-9f79-42d0-b3ba-4cf5176cbec1 tempest-VolumesSnapshotListTestJSON-276023532 None] Acquiring lock "cinder-4d6cccdd-21ad-45e6-aa71-561350ab8f54-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:44.724555 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b04dd203-9f79-42d0-b3ba-4cf5176cbec1 tempest-VolumesSnapshotListTestJSON-276023532 None] Lock "cinder-4d6cccdd-21ad-45e6-aa71-561350ab8f54-delete_snapshot" acquired by "delete_snapshot" :: waited 0.012s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:44.726119 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b04dd203-9f79-42d0-b3ba-4cf5176cbec1 tempest-VolumesSnapshotListTestJSON-276023532 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:45.469723 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-ff723f48-a3bc-4a4e-b88b-54238aeef968 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Create snapshot completed successfully Sep 27 10:37:45.490493 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-b04dd203-9f79-42d0-b3ba-4cf5176cbec1 tempest-VolumesSnapshotListTestJSON-276023532 None] Created reservations ['bed23003-b0f3-4cf9-b6e1-1c0470ac7b3e', '3c37d476-0dc9-4d44-803b-76db0e43c382', '3b6e1e9f-0138-407e-b3cf-586e15c31bae', '4613baca-afd3-4247-acc2-0b102b358ea0'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:45.528411 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b04dd203-9f79-42d0-b3ba-4cf5176cbec1 tempest-VolumesSnapshotListTestJSON-276023532 None] Delete snapshot completed successfully. Sep 27 10:37:45.532920 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-b04dd203-9f79-42d0-b3ba-4cf5176cbec1 tempest-VolumesSnapshotListTestJSON-276023532 None] Lock "cinder-4d6cccdd-21ad-45e6-aa71-561350ab8f54-delete_snapshot" released by "delete_snapshot" :: held 0.809s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:45.818736 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a739b1aa-9ec0-402e-a19e-6e59da3724ac tempest-VolumesSnapshotListTestJSON-276023532 None] Acquiring lock "cinder-ad517549-c4d5-48e5-934e-5eb18da3c34f-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:45.829132 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a739b1aa-9ec0-402e-a19e-6e59da3724ac tempest-VolumesSnapshotListTestJSON-276023532 None] Lock "cinder-ad517549-c4d5-48e5-934e-5eb18da3c34f-delete_snapshot" acquired by "delete_snapshot" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:45.831060 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a739b1aa-9ec0-402e-a19e-6e59da3724ac tempest-VolumesSnapshotListTestJSON-276023532 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:46.070120 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6645ac28-f0d4-49ea-91fd-9fb76e91fb60 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Acquiring lock "cinder-776552cc-3472-4637-8985-12b1bdf5e15a-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:46.081345 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6645ac28-f0d4-49ea-91fd-9fb76e91fb60 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Lock "cinder-776552cc-3472-4637-8985-12b1bdf5e15a-delete_snapshot" acquired by "delete_snapshot" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:46.082853 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6645ac28-f0d4-49ea-91fd-9fb76e91fb60 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:46.532818 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-a739b1aa-9ec0-402e-a19e-6e59da3724ac tempest-VolumesSnapshotListTestJSON-276023532 None] Created reservations ['34518f5d-425e-4128-8053-a0897105a94f', '959006f0-65f3-451b-ad85-6938f76bd2f2', 'f21b5817-a4ae-44b3-aa42-61e046540f49', 'd9f3b936-b128-4574-b370-5c1bc29a8def'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:46.538804 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-6645ac28-f0d4-49ea-91fd-9fb76e91fb60 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Created reservations ['699d9c8c-455e-48e0-b631-40da48e863c6', 'd2406fef-895a-4a52-9285-ef37f91b5fe4', 'ba5fc640-ccd0-40c0-acd2-316da0cb9efd', 'ca96861c-3450-4ef6-a4d8-a57ae954000b'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:46.593608 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-a739b1aa-9ec0-402e-a19e-6e59da3724ac tempest-VolumesSnapshotListTestJSON-276023532 None] Delete snapshot completed successfully. Sep 27 10:37:46.596677 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6645ac28-f0d4-49ea-91fd-9fb76e91fb60 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Delete snapshot completed successfully. Sep 27 10:37:46.599023 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a739b1aa-9ec0-402e-a19e-6e59da3724ac tempest-VolumesSnapshotListTestJSON-276023532 None] Lock "cinder-ad517549-c4d5-48e5-934e-5eb18da3c34f-delete_snapshot" released by "delete_snapshot" :: held 0.770s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:46.602671 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6645ac28-f0d4-49ea-91fd-9fb76e91fb60 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Lock "cinder-776552cc-3472-4637-8985-12b1bdf5e15a-delete_snapshot" released by "delete_snapshot" :: held 0.521s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:46.943606 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-02155a8b-f760-4131-8248-24ece8dbb459 tempest-VolumesSnapshotListTestJSON-276023532 None] Acquiring lock "cinder-51f6f412-0556-42cb-848a-450366daa13a-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:46.953270 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-02155a8b-f760-4131-8248-24ece8dbb459 tempest-VolumesSnapshotListTestJSON-276023532 None] Lock "cinder-51f6f412-0556-42cb-848a-450366daa13a-delete_snapshot" acquired by "delete_snapshot" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:46.955059 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-02155a8b-f760-4131-8248-24ece8dbb459 tempest-VolumesSnapshotListTestJSON-276023532 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:47.212831 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Acquiring lock "cinder-b8e29dbc-206d-4092-ac49-441ace7c9012-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:47.227875 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Lock "cinder-b8e29dbc-206d-4092-ac49-441ace7c9012-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:47.231721 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=59,cinder:UPDATE=30,cinder:DELETE=5,cinder:INSERT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:47.269180 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:47.307423 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:37:47.307957 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Volume volume-b8e29dbc-206d-4092-ac49-441ace7c9012 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:47.313032 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] deleting rbd volume volume-b8e29dbc-206d-4092-ac49-441ace7c9012 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:37:47.511592 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-02155a8b-f760-4131-8248-24ece8dbb459 tempest-VolumesSnapshotListTestJSON-276023532 None] Created reservations ['f61e5a9a-e198-453c-b255-82fe1ba60128', '03a9255f-b1f7-4387-b01e-621249150d84', '55de4a68-2dfb-4676-802d-4fec1e068248', 'bd114396-11cb-4780-81f9-47f89b1945b9'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:47.544803 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=66,cinder:UPDATE=24,cinder:INSERT=10 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:47.557041 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Created reservations ['d30e03f8-166a-4156-b94d-7cae249d5fe4', 'd1e7850b-3a53-41a8-a407-c3b7046d3a02', '0da5da07-87f0-4303-9ecc-6d69ca8785a0', '45430e26-4ed8-4e46-b67f-74e43aa375c6'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:47.595821 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-02155a8b-f760-4131-8248-24ece8dbb459 tempest-VolumesSnapshotListTestJSON-276023532 None] Delete snapshot completed successfully. Sep 27 10:37:47.603475 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-02155a8b-f760-4131-8248-24ece8dbb459 tempest-VolumesSnapshotListTestJSON-276023532 None] Lock "cinder-51f6f412-0556-42cb-848a-450366daa13a-delete_snapshot" released by "delete_snapshot" :: held 0.650s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:47.628416 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:47.656075 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:47.681327 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:47.689015 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Deleted volume successfully. Sep 27 10:37:47.703135 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-9f3944bf-9ab2-4d29-b33c-0c2aa03cb4cf tempest-VolumeSnapshotQuotasNegativeTestJSON-2015984561 None] Lock "cinder-b8e29dbc-206d-4092-ac49-441ace7c9012-delete_volume" released by "delete_volume" :: held 0.476s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:48.054814 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] Acquiring lock "cinder-6a3f4155-55b2-4aae-8e1f-96102d5111fc-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:48.063451 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] Lock "cinder-6a3f4155-55b2-4aae-8e1f-96102d5111fc-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:48.098746 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:48.136556 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:37:48.137017 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] Volume volume-6a3f4155-55b2-4aae-8e1f-96102d5111fc is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:48.142569 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] deleting rbd volume volume-6a3f4155-55b2-4aae-8e1f-96102d5111fc {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:37:48.341411 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] Created reservations ['cada254e-1412-414e-9bd1-d1c2f419a8ca', 'fd767fb4-66c0-4ef7-8ed7-0ed0d88efb87', 'fc90603c-9f10-4639-920b-bd3e41e3e54c', 'dcff26fe-7f50-403e-bb29-8e8410f1cd21'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:48.390688 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:48.425001 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:48.454415 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:48.458843 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] Deleted volume successfully. Sep 27 10:37:48.475456 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e1befb1a-beb2-4a98-bed2-a06e7506f0dd tempest-VolumesSnapshotListTestJSON-276023532 None] Lock "cinder-6a3f4155-55b2-4aae-8e1f-96102d5111fc-delete_volume" released by "delete_volume" :: held 0.412s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:51.149394 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:37:51.167530 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Flow 'volume_create_manager' (4afe9409-d612-4053-8257-9ff992cc7713) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:51.173674 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (22a7a133-b8a7-4d1f-bad2-e35f0c57980e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:51.189619 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (22a7a133-b8a7-4d1f-bad2-e35f0c57980e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:37:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotNegativeTestJSON-Volume-332293210',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=62e70e7e-7803-4d09-a984-2c01200e7921,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='dcb31ee64b334635bd6fcc7b8632faca',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:37:51Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:37:51Z,use_quota=True,user_id='c3524dd067b54dc69691738921a83bfc',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:51.195083 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c840d6a2-d557-414d-b018-0cdec96eb5e9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:51.198977 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c840d6a2-d557-414d-b018-0cdec96eb5e9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:51.203744 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (eeb763d9-4f4e-45f7-99f4-968ed26057dc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:51.207550 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (eeb763d9-4f4e-45f7-99f4-968ed26057dc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '62e70e7e-7803-4d09-a984-2c01200e7921', 'volume_name': 'volume-62e70e7e-7803-4d09-a984-2c01200e7921', 'volume_size': 2}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:51.212411 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d75c3da0-6cae-4bb3-9cf8-9c70edf43cb7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:51.216495 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d75c3da0-6cae-4bb3-9cf8-9c70edf43cb7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:51.220925 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4dac8297-f77c-432e-8f87-0595e48998f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:51.221968 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Volume 62e70e7e-7803-4d09-a984-2c01200e7921: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-62e70e7e-7803-4d09-a984-2c01200e7921', 'volume_size': 2} Sep 27 10:37:51.222353 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] creating volume 'volume-62e70e7e-7803-4d09-a984-2c01200e7921' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:37:51.223119 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:51.291877 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:37:51.292244 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:37:51.296300 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4dac8297-f77c-432e-8f87-0595e48998f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-62e70e7e-7803-4d09-a984-2c01200e7921', 'volume_size': 2}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:51.300767 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1206f600-4c43-464f-b0d8-0f735824b2dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:37:51.310944 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Volume volume-62e70e7e-7803-4d09-a984-2c01200e7921 (62e70e7e-7803-4d09-a984-2c01200e7921): created successfully Sep 27 10:37:51.314533 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1206f600-4c43-464f-b0d8-0f735824b2dd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:37:51.320148 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Flow 'volume_create_manager' (4afe9409-d612-4053-8257-9ff992cc7713) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:37:51.329773 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-cd7a92e0-d4c2-4b49-b4ad-8fdd45e2899f tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Created volume successfully. Sep 27 10:37:52.412806 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-54052d28-0b8f-4ae6-a32c-62e9ee2efe81 tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:53.562333 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-54052d28-0b8f-4ae6-a32c-62e9ee2efe81 tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Create snapshot completed successfully Sep 27 10:37:54.568431 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7af14cb1-e4e6-446b-8293-35ed737eb4bc tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Acquiring lock "cinder-90f87b9c-9ed0-4791-899b-ce6d2e329f39-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:54.579630 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7af14cb1-e4e6-446b-8293-35ed737eb4bc tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Lock "cinder-90f87b9c-9ed0-4791-899b-ce6d2e329f39-delete_snapshot" acquired by "delete_snapshot" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:54.580149 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-7af14cb1-e4e6-446b-8293-35ed737eb4bc tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:55.584244 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-7af14cb1-e4e6-446b-8293-35ed737eb4bc tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Created reservations ['47ee321c-9d5b-4b8e-97cc-890e7f88c996', '7fd31cdb-49ba-4a47-b26b-c70308f8651a', '3cb730eb-6970-46c9-973d-8fbe4f2f9d85', '69817bab-dec6-42b7-a71a-10eb1ede5815'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:55.634632 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-7af14cb1-e4e6-446b-8293-35ed737eb4bc tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Delete snapshot completed successfully. Sep 27 10:37:55.640537 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-7af14cb1-e4e6-446b-8293-35ed737eb4bc tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Lock "cinder-90f87b9c-9ed0-4791-899b-ce6d2e329f39-delete_snapshot" released by "delete_snapshot" :: held 1.062s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:37:55.696997 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Acquiring lock "cinder-62e70e7e-7803-4d09-a984-2c01200e7921-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:37:55.707222 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Lock "cinder-62e70e7e-7803-4d09-a984-2c01200e7921-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:37:55.753766 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=54,cinder:INSERT=8,cinder:UPDATE=37,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:55.753766 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:55.805927 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:37:55.806361 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Volume volume-62e70e7e-7803-4d09-a984-2c01200e7921 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:37:55.814167 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] deleting rbd volume volume-62e70e7e-7803-4d09-a984-2c01200e7921 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:37:56.053104 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Created reservations ['13fcf2ea-badf-4007-8c0a-64a1dc9286ca', 'c96fd611-2afe-41b9-9727-a3fe3f09b624', 'cdd2908a-9e3d-4ade-bcf2-d7196c875461', '8a5d3053-ce0a-424f-8175-d545c80e7b5b'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:37:56.099426 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:56.125753 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:37:56.156739 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:37:56.161142 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Deleted volume successfully. Sep 27 10:37:56.174932 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-70230774-2783-4ffa-8936-7119d8f7e91e tempest-VolumesSnapshotNegativeTestJSON-1969417607 None] Lock "cinder-62e70e7e-7803-4d09-a984-2c01200e7921-delete_volume" released by "delete_volume" :: held 0.468s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:38:07.660705 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=14,cinder:INSERT=1,cinder:UPDATE=8 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:10.377614 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:38:10.395843 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (b0a0928d-f313-4c0a-99b4-8fb193d2fb2c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:10.402058 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6a3d7e25-bc73-4981-93c2-5cc01743fdf9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:10.418072 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6a3d7e25-bc73-4981-93c2-5cc01743fdf9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:38:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-873339140',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=24403991-c005-4f9c-8819-037884983563,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='60f3e5639c294a53899e29af88c9c38b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:38:10Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:38:10Z,use_quota=True,user_id='88dbdf0286c34c179364a772529dbfe8',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:10.423115 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3013d8a4-3b33-4a83-92f2-b9f763a5c870) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:10.427020 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3013d8a4-3b33-4a83-92f2-b9f763a5c870) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:10.431886 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bcd49fd3-68e6-46b5-a173-57d7bab2f7af) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:10.435757 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bcd49fd3-68e6-46b5-a173-57d7bab2f7af) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '24403991-c005-4f9c-8819-037884983563', 'volume_name': 'volume-24403991-c005-4f9c-8819-037884983563', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:10.440372 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (470f1fc0-6960-4af1-a77b-e12dd3ba2a11) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:10.444420 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (470f1fc0-6960-4af1-a77b-e12dd3ba2a11) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:10.448887 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a459cec3-6786-4f1b-a1e6-98fc0f0a1a0d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:10.450015 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Volume 24403991-c005-4f9c-8819-037884983563: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-24403991-c005-4f9c-8819-037884983563', 'volume_size': 1} Sep 27 10:38:10.450416 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] creating volume 'volume-24403991-c005-4f9c-8819-037884983563' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:38:10.451145 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:10.512789 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:38:10.513054 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:38:10.516581 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a459cec3-6786-4f1b-a1e6-98fc0f0a1a0d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-24403991-c005-4f9c-8819-037884983563', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:10.520998 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ef7b9689-bde0-414c-a822-de7ed8fc6495) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:10.533450 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Volume volume-24403991-c005-4f9c-8819-037884983563 (24403991-c005-4f9c-8819-037884983563): created successfully Sep 27 10:38:10.536824 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ef7b9689-bde0-414c-a822-de7ed8fc6495) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:10.542049 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (b0a0928d-f313-4c0a-99b4-8fb193d2fb2c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:10.551483 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-c87b2c3c-0088-42c0-a1ae-a7a8423b5534 tempest-VolumesActionsTest-1137661699 None] Created volume successfully. Sep 27 10:38:11.436072 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-717ca245-3dee-426e-be56-240652ef99b9 tempest-VolumesActionsTest-1137661699 None] Acquiring lock "cinder-24403991-c005-4f9c-8819-037884983563" by "attach_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:38:11.445780 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-717ca245-3dee-426e-be56-240652ef99b9 tempest-VolumesActionsTest-1137661699 None] Lock "cinder-24403991-c005-4f9c-8819-037884983563" acquired by "attach_volume" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:38:11.488303 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-717ca245-3dee-426e-be56-240652ef99b9 tempest-VolumesActionsTest-1137661699 None] Attaching volume 24403991-c005-4f9c-8819-037884983563 to instance 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb at mountpoint /dev/sdb on host None. Sep 27 10:38:11.495088 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=31,cinder:SELECT=57,cinder:DELETE=4,cinder:INSERT=8 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:11.517144 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-717ca245-3dee-426e-be56-240652ef99b9 tempest-VolumesActionsTest-1137661699 None] Attach volume completed successfully. Sep 27 10:38:11.522723 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-717ca245-3dee-426e-be56-240652ef99b9 tempest-VolumesActionsTest-1137661699 None] Lock "cinder-24403991-c005-4f9c-8819-037884983563" released by "attach_volume" :: held 0.077s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:38:12.030214 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0893a766-f96c-4f30-9cd4-2ba31866ac63 tempest-VolumesActionsTest-287929529 None] Terminate volume connection completed successfully. Sep 27 10:38:12.051825 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0893a766-f96c-4f30-9cd4-2ba31866ac63 tempest-VolumesActionsTest-287929529 None] Acquiring lock "cinder-24403991-c005-4f9c-8819-037884983563-detach_volume" by "detach_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:38:12.063003 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0893a766-f96c-4f30-9cd4-2ba31866ac63 tempest-VolumesActionsTest-287929529 None] Lock "cinder-24403991-c005-4f9c-8819-037884983563-detach_volume" acquired by "detach_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:38:12.076552 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0893a766-f96c-4f30-9cd4-2ba31866ac63 tempest-VolumesActionsTest-287929529 None] Detaching volume 24403991-c005-4f9c-8819-037884983563 from instance 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb. Sep 27 10:38:12.109951 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0893a766-f96c-4f30-9cd4-2ba31866ac63 tempest-VolumesActionsTest-287929529 None] Detach volume completed successfully. Sep 27 10:38:12.115824 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0893a766-f96c-4f30-9cd4-2ba31866ac63 tempest-VolumesActionsTest-287929529 None] Lock "cinder-24403991-c005-4f9c-8819-037884983563-detach_volume" released by "detach_volume" :: held 0.053s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:38:12.281309 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6df475c4-ccb1-404f-9f24-ec66e653acad tempest-VolumesActionsTest-1137661699 None] Acquiring lock "cinder-24403991-c005-4f9c-8819-037884983563-detach_volume" by "detach_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:38:12.290297 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6df475c4-ccb1-404f-9f24-ec66e653acad tempest-VolumesActionsTest-1137661699 None] Lock "cinder-24403991-c005-4f9c-8819-037884983563-detach_volume" acquired by "detach_volume" :: waited 0.009s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:38:12.290913 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6df475c4-ccb1-404f-9f24-ec66e653acad tempest-VolumesActionsTest-1137661699 None] Volume detach called, but volume not attached. Sep 27 10:38:12.303503 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-6df475c4-ccb1-404f-9f24-ec66e653acad tempest-VolumesActionsTest-1137661699 None] Lock "cinder-24403991-c005-4f9c-8819-037884983563-detach_volume" released by "detach_volume" :: held 0.013s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:38:15.328109 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:38:15.347098 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (73d6852b-a105-4cc0-bde0-ce4818acb7f1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:15.353849 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (573c9337-6582-4ad8-bb23-daba4b40dcb3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:15.369647 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (573c9337-6582-4ad8-bb23-daba4b40dcb3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:38:15Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-334802424',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=815404db-1e11-4551-9dd4-c1981e0a4fa8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='60f3e5639c294a53899e29af88c9c38b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:38:15Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:38:15Z,use_quota=True,user_id='88dbdf0286c34c179364a772529dbfe8',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:15.374779 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9d18719c-4a5d-41fd-a68d-11053eb501f4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:15.378748 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9d18719c-4a5d-41fd-a68d-11053eb501f4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:15.383855 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a300dee0-56d1-4481-b508-4cee516a7856) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:15.387734 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a300dee0-56d1-4481-b508-4cee516a7856) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '815404db-1e11-4551-9dd4-c1981e0a4fa8', 'volume_name': 'volume-815404db-1e11-4551-9dd4-c1981e0a4fa8', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:15.392372 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a0959c4a-4389-4e45-81ec-6d1f9a2d3cce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:15.396102 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a0959c4a-4389-4e45-81ec-6d1f9a2d3cce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:15.400572 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (75ef9d60-2891-4dc3-979d-1f26bbe20407) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:15.401635 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Volume 815404db-1e11-4551-9dd4-c1981e0a4fa8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-815404db-1e11-4551-9dd4-c1981e0a4fa8', 'volume_size': 1} Sep 27 10:38:15.402026 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] creating volume 'volume-815404db-1e11-4551-9dd4-c1981e0a4fa8' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:38:15.402663 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:15.457146 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:38:15.457519 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:38:15.461248 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (75ef9d60-2891-4dc3-979d-1f26bbe20407) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-815404db-1e11-4551-9dd4-c1981e0a4fa8', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:15.465643 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0c23e204-e4c1-48ce-bf78-412fec3502e2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:15.477539 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Volume volume-815404db-1e11-4551-9dd4-c1981e0a4fa8 (815404db-1e11-4551-9dd4-c1981e0a4fa8): created successfully Sep 27 10:38:15.481104 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0c23e204-e4c1-48ce-bf78-412fec3502e2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:15.486370 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (73d6852b-a105-4cc0-bde0-ce4818acb7f1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:15.497021 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6c0236ad-49b5-4719-b438-040f9b267bc3 tempest-VolumesActionsTest-1137661699 None] Created volume successfully. Sep 27 10:38:16.534196 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] Acquiring lock "cinder-815404db-1e11-4551-9dd4-c1981e0a4fa8-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:38:16.546702 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] Lock "cinder-815404db-1e11-4551-9dd4-c1981e0a4fa8-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:38:16.579291 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:16.634106 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:38:16.634530 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] Volume volume-815404db-1e11-4551-9dd4-c1981e0a4fa8 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:38:16.640108 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] deleting rbd volume volume-815404db-1e11-4551-9dd4-c1981e0a4fa8 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:38:16.837817 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] Created reservations ['51c8c41e-aea7-4791-bde9-56f43d3a411b', 'f2df1132-388a-4c8f-9b80-d8282079b30b', '135392a0-e91a-40c3-b7ea-cfff91a58fb4', '93b6c3e4-7a84-49e9-ae50-058c79e80837'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:38:16.884791 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:16.908581 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:16.932744 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:38:16.936876 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] Deleted volume successfully. Sep 27 10:38:16.950895 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-d6c089a6-5774-481e-8e53-85ee7f3d54ae tempest-VolumesActionsTest-287929529 None] Lock "cinder-815404db-1e11-4551-9dd4-c1981e0a4fa8-delete_volume" released by "delete_volume" :: held 0.404s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:38:17.959854 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:38:17.980357 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (fb7f4516-7e51-4d94-a35d-e4ce9debadd5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:17.986603 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a9f1a8f8-0afc-4e0c-a69c-28320356b261) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:18.002387 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a9f1a8f8-0afc-4e0c-a69c-28320356b261) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:38:18Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-702206797',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=53aad74d-b4d3-4131-bde5-d3e7f1692f1c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='60f3e5639c294a53899e29af88c9c38b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:38:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:38:18Z,use_quota=True,user_id='88dbdf0286c34c179364a772529dbfe8',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:18.007368 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3656b67a-7743-4ac6-bbef-7053dfde8912) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:18.011324 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3656b67a-7743-4ac6-bbef-7053dfde8912) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:18.016065 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b82405a0-3f64-4a5e-8fe1-3254ad323a0e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:18.019882 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b82405a0-3f64-4a5e-8fe1-3254ad323a0e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '53aad74d-b4d3-4131-bde5-d3e7f1692f1c', 'volume_name': 'volume-53aad74d-b4d3-4131-bde5-d3e7f1692f1c', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:18.024645 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (847cadeb-4a02-49a8-a547-605b71a3e3f0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:18.028305 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (847cadeb-4a02-49a8-a547-605b71a3e3f0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:18.032869 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1a8050f8-c31d-4613-9076-a7bf4de236ab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:18.033881 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Volume 53aad74d-b4d3-4131-bde5-d3e7f1692f1c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-53aad74d-b4d3-4131-bde5-d3e7f1692f1c', 'volume_size': 1} Sep 27 10:38:18.034265 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] creating volume 'volume-53aad74d-b4d3-4131-bde5-d3e7f1692f1c' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:38:18.035347 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:18.096963 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:38:18.097328 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:38:18.100926 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1a8050f8-c31d-4613-9076-a7bf4de236ab) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-53aad74d-b4d3-4131-bde5-d3e7f1692f1c', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:18.105380 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (40432aa4-6166-46f8-98de-5ee7d4e8a834) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:18.117117 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Volume volume-53aad74d-b4d3-4131-bde5-d3e7f1692f1c (53aad74d-b4d3-4131-bde5-d3e7f1692f1c): created successfully Sep 27 10:38:18.120797 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (40432aa4-6166-46f8-98de-5ee7d4e8a834) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:18.126111 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (fb7f4516-7e51-4d94-a35d-e4ce9debadd5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:18.136544 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-2ef84bf5-9898-4ad7-9a0e-db82e74a98c5 tempest-VolumesActionsTest-1137661699 None] Created volume successfully. Sep 27 10:38:19.160487 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] Acquiring lock "cinder-53aad74d-b4d3-4131-bde5-d3e7f1692f1c-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:38:19.171166 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] Lock "cinder-53aad74d-b4d3-4131-bde5-d3e7f1692f1c-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:38:19.200337 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:19.246280 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:38:19.246687 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] Volume volume-53aad74d-b4d3-4131-bde5-d3e7f1692f1c is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:38:19.253395 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] deleting rbd volume volume-53aad74d-b4d3-4131-bde5-d3e7f1692f1c {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:38:19.460997 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] Created reservations ['92b08ec7-72cc-4533-85f1-4ac5ca898440', '5e073f23-d1e6-4f4f-b806-827b83509b8e', '1cc084d2-33bb-41c7-8ce9-316b2e39661f', '98354ef0-b763-4c2a-b990-e8b17eed6919'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:38:19.519689 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:19.547612 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:19.572513 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:38:19.576403 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] Deleted volume successfully. Sep 27 10:38:19.591040 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-3b61ef7d-0464-44ed-8a94-c21944d4b524 tempest-VolumesActionsTest-287929529 None] Lock "cinder-53aad74d-b4d3-4131-bde5-d3e7f1692f1c-delete_volume" released by "delete_volume" :: held 0.420s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:38:20.583788 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:38:20.602141 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (87d723a2-7837-4610-8449-82a030864da8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:20.608504 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7075af0a-33b5-4257-b074-16d8ec5e96e4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:20.625338 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7075af0a-33b5-4257-b074-16d8ec5e96e4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:38:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-172423502',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=91bcf131-0012-4e46-bfad-f8162448f625,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='60f3e5639c294a53899e29af88c9c38b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:38:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:38:21Z,use_quota=True,user_id='88dbdf0286c34c179364a772529dbfe8',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:20.630351 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (364976a5-10d9-4e7e-9252-e68192daf5d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:20.634248 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (364976a5-10d9-4e7e-9252-e68192daf5d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:20.639200 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6076b450-8c8d-4fad-9ba6-bdf116d8a730) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:20.643045 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6076b450-8c8d-4fad-9ba6-bdf116d8a730) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '91bcf131-0012-4e46-bfad-f8162448f625', 'volume_name': 'volume-91bcf131-0012-4e46-bfad-f8162448f625', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:20.647618 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a1e488b0-1d50-44b1-a4f5-27fff4d34cce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:20.651539 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a1e488b0-1d50-44b1-a4f5-27fff4d34cce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:20.655915 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e0d0a17d-11af-4169-a11a-135f56fb1f85) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:20.656968 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Volume 91bcf131-0012-4e46-bfad-f8162448f625: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-91bcf131-0012-4e46-bfad-f8162448f625', 'volume_size': 1} Sep 27 10:38:20.657352 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] creating volume 'volume-91bcf131-0012-4e46-bfad-f8162448f625' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:38:20.658104 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:20.719929 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:38:20.720298 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:38:20.723900 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e0d0a17d-11af-4169-a11a-135f56fb1f85) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-91bcf131-0012-4e46-bfad-f8162448f625', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:20.728501 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0184ed67-3174-4da3-812d-5c93be1ffd4d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:20.739899 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Volume volume-91bcf131-0012-4e46-bfad-f8162448f625 (91bcf131-0012-4e46-bfad-f8162448f625): created successfully Sep 27 10:38:20.743485 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0184ed67-3174-4da3-812d-5c93be1ffd4d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:20.748733 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (87d723a2-7837-4610-8449-82a030864da8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:20.760964 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-fdffe954-c475-4eea-9072-91b018ef7fb9 tempest-VolumesActionsTest-1137661699 None] Created volume successfully. Sep 27 10:38:21.775763 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] Acquiring lock "cinder-91bcf131-0012-4e46-bfad-f8162448f625-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:38:21.786853 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] Lock "cinder-91bcf131-0012-4e46-bfad-f8162448f625-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:38:21.813693 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:21.856985 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:38:21.857408 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] Volume volume-91bcf131-0012-4e46-bfad-f8162448f625 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:38:21.862265 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] deleting rbd volume volume-91bcf131-0012-4e46-bfad-f8162448f625 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:38:22.071717 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] Created reservations ['69cb5aa3-bd68-4ff0-888b-0215269d7975', '5a6e12c1-c19c-45b1-9100-b0bb95bf2621', '2553036f-c030-4b34-9d3f-720a2da232e0', '85c33ab3-90b3-49e6-86a4-e5c36f139377'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:38:22.087270 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=62,cinder:UPDATE=28,cinder:DELETE=3,cinder:INSERT=7 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:22.122764 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:22.147467 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:22.172466 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:38:22.176630 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] Deleted volume successfully. Sep 27 10:38:22.191611 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-93ff259e-62b3-43c5-8e5c-51278971726a tempest-VolumesActionsTest-287929529 None] Lock "cinder-91bcf131-0012-4e46-bfad-f8162448f625-delete_volume" released by "delete_volume" :: held 0.405s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:38:23.214799 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:38:23.235261 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (47497a30-d305-475c-b538-6b021819d719) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:23.241384 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0086eb40-40a8-4d30-8420-2bfde2d4aa46) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:23.244297 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=55,cinder:UPDATE=35,cinder:INSERT=6,cinder:DELETE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:23.258320 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0086eb40-40a8-4d30-8420-2bfde2d4aa46) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:38:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-745584511',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='60f3e5639c294a53899e29af88c9c38b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:38:23Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:38:23Z,use_quota=True,user_id='88dbdf0286c34c179364a772529dbfe8',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:23.263574 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5190fe3d-fc12-4518-9319-ce9f35a2bd34) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:23.267564 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5190fe3d-fc12-4518-9319-ce9f35a2bd34) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:23.272401 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (18be71b0-7581-4a44-8bcb-8b6525ae4869) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:23.276355 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (18be71b0-7581-4a44-8bcb-8b6525ae4869) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9', 'volume_name': 'volume-f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:23.281120 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8105afc6-d1e6-4037-97d1-62c06cfc9c48) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:23.284874 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8105afc6-d1e6-4037-97d1-62c06cfc9c48) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:23.289404 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2282eb95-b227-4ce5-a6c0-a29fac8bab15) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:23.290455 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Volume f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9', 'volume_size': 1} Sep 27 10:38:23.290847 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] creating volume 'volume-f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:38:23.291604 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:23.353840 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:38:23.354323 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:38:23.358109 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2282eb95-b227-4ce5-a6c0-a29fac8bab15) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:23.362522 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (31460b36-bce3-420a-9302-29d05635689f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:23.374946 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Volume volume-f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9 (f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9): created successfully Sep 27 10:38:23.378310 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (31460b36-bce3-420a-9302-29d05635689f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:23.383518 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (47497a30-d305-475c-b538-6b021819d719) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:23.393555 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-dd401b62-4bf6-432c-9a84-fcfd10626e34 tempest-VolumesActionsTest-1137661699 None] Created volume successfully. Sep 27 10:38:24.398996 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] Acquiring lock "cinder-f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:38:24.409264 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] Lock "cinder-f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:38:24.438921 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:24.482192 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:38:24.482612 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] Volume volume-f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:38:24.488598 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] deleting rbd volume volume-f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:38:24.709514 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] Created reservations ['45b93241-f6a5-4c3e-bad4-ee44efe585b8', '14ecca9c-486e-418e-b06f-49a0ff43c09e', '03288024-cd72-4f70-b232-dd901fe291cb', '9b2cac66-ccbf-4f32-b7c6-6bf92c397911'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:38:24.759164 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:24.785289 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:24.814127 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:38:24.819245 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] Deleted volume successfully. Sep 27 10:38:24.834642 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e37acfbf-617f-4840-a759-14463d05bd0c tempest-VolumesActionsTest-287929529 None] Lock "cinder-f7cd5c37-93d7-4c23-9acc-8b5000c5f0f9-delete_volume" released by "delete_volume" :: held 0.425s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:38:25.823357 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:38:25.842794 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (27e749f6-9388-40db-9420-e3d8bdbe8546) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:25.848915 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4bce8240-767e-4ce7-a364-b43ead6c4ad9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:25.865988 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4bce8240-767e-4ce7-a364-b43ead6c4ad9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:38:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-669620888',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e1d71e23-258f-45de-86cc-e80a14120c53,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='60f3e5639c294a53899e29af88c9c38b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:38:26Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:38:26Z,use_quota=True,user_id='88dbdf0286c34c179364a772529dbfe8',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:25.871006 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (944d4e8d-0e81-45d8-a488-b5e3dd90f6e4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:25.875306 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (944d4e8d-0e81-45d8-a488-b5e3dd90f6e4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:25.880051 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8e41d48b-d88a-40f7-925d-d7a74107580c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:25.883876 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8e41d48b-d88a-40f7-925d-d7a74107580c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e1d71e23-258f-45de-86cc-e80a14120c53', 'volume_name': 'volume-e1d71e23-258f-45de-86cc-e80a14120c53', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:25.888510 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cd556506-48da-4a4e-92ba-e0013758e7e3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:25.892183 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cd556506-48da-4a4e-92ba-e0013758e7e3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:25.896550 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (43f042ac-1f25-4e41-a3aa-a129c5d20dee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:25.897593 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Volume e1d71e23-258f-45de-86cc-e80a14120c53: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e1d71e23-258f-45de-86cc-e80a14120c53', 'volume_size': 1} Sep 27 10:38:25.897977 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] creating volume 'volume-e1d71e23-258f-45de-86cc-e80a14120c53' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:38:25.898798 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:25.960598 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:38:25.961001 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:38:25.964643 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (43f042ac-1f25-4e41-a3aa-a129c5d20dee) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e1d71e23-258f-45de-86cc-e80a14120c53', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:25.969131 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (40d8be84-8990-4f11-8bcd-29dbf73ccf9d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:25.980813 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Volume volume-e1d71e23-258f-45de-86cc-e80a14120c53 (e1d71e23-258f-45de-86cc-e80a14120c53): created successfully Sep 27 10:38:25.984652 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (40d8be84-8990-4f11-8bcd-29dbf73ccf9d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:25.989893 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Flow 'volume_create_manager' (27e749f6-9388-40db-9420-e3d8bdbe8546) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:25.999268 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-eed65218-69f8-4b1b-87e7-40bc1137b279 tempest-VolumesActionsTest-1137661699 None] Created volume successfully. Sep 27 10:38:27.397779 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] Acquiring lock "cinder-e1d71e23-258f-45de-86cc-e80a14120c53-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:38:27.409560 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] Lock "cinder-e1d71e23-258f-45de-86cc-e80a14120c53-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:38:27.442333 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:27.481032 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:38:27.481473 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] Volume volume-e1d71e23-258f-45de-86cc-e80a14120c53 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:38:27.486958 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] deleting rbd volume volume-e1d71e23-258f-45de-86cc-e80a14120c53 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:38:27.696532 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] Created reservations ['c19b9f47-e086-4933-bff9-03654b9705f8', '83a67a79-aa06-42da-b9c0-d856729f2cde', 'd61c68de-8437-4fc8-ab51-78c8e28414ff', '94760bf0-d502-4de2-a162-9fed05ad1858'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:38:27.742503 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:27.768038 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:27.792601 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:38:27.796772 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] Deleted volume successfully. Sep 27 10:38:27.810367 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-0d787cc4-35aa-4ce5-a275-5bf9d5b0e02c tempest-VolumesActionsTest-1137661699 None] Lock "cinder-e1d71e23-258f-45de-86cc-e80a14120c53-delete_volume" released by "delete_volume" :: held 0.401s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:38:28.649617 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] Acquiring lock "cinder-24403991-c005-4f9c-8819-037884983563-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:38:28.663222 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] Lock "cinder-24403991-c005-4f9c-8819-037884983563-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:38:28.707711 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:28.746167 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:38:28.746717 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] Volume volume-24403991-c005-4f9c-8819-037884983563 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:38:28.753888 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] deleting rbd volume volume-24403991-c005-4f9c-8819-037884983563 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:38:28.955224 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] Created reservations ['390efc7a-22dd-4dca-958f-196e0dc7c7ef', 'c5ffd140-69c6-455b-8101-d8538c454147', '7f5bb208-79eb-4b51-9549-c1b2a8596b13', 'b9f9b1ef-c1d6-4ecd-89d9-9b078c14cb56'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:38:29.000968 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:29.025439 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:29.050886 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:38:29.055184 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] Deleted volume successfully. Sep 27 10:38:29.069583 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-2116fd46-c1cc-4af7-89df-a1fb69d7353d tempest-VolumesActionsTest-1137661699 None] Lock "cinder-24403991-c005-4f9c-8819-037884983563-delete_volume" released by "delete_volume" :: held 0.407s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:38:30.020986 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:38:30.021740 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:30.048369 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:30.075297 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:38:32.952162 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:38:32.971180 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Flow 'volume_create_manager' (9f2216d4-2ed1-4c19-b8c1-2dbc21f9e620) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:32.977323 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (90cacc49-1d11-4720-b0a3-ef48f25f3980) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:32.993482 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (90cacc49-1d11-4720-b0a3-ef48f25f3980) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:38:33Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Volume-323066426',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=66378b2c-0be3-45e8-8692-38c639675c8c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3de8116c00564abd82f93555f3f81993',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:38:33Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:38:33Z,use_quota=True,user_id='64c73ef93e3b4d8da546f4ac18520560',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:32.998509 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ab0e8526-bcbe-4ca0-a202-dce18284ff7e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:33.002498 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ab0e8526-bcbe-4ca0-a202-dce18284ff7e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:33.007353 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a2367225-f3ad-462c-9689-1293f0c5dfc5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:33.011175 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a2367225-f3ad-462c-9689-1293f0c5dfc5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '66378b2c-0be3-45e8-8692-38c639675c8c', 'volume_name': 'volume-66378b2c-0be3-45e8-8692-38c639675c8c', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:33.015907 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cfe4e813-8d8c-4c58-9f31-673ee82d927b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:33.019636 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cfe4e813-8d8c-4c58-9f31-673ee82d927b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:33.024090 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5f061bb9-cb4a-44ad-83ef-ff042520bec5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:33.025153 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Volume 66378b2c-0be3-45e8-8692-38c639675c8c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-66378b2c-0be3-45e8-8692-38c639675c8c', 'volume_size': 1} Sep 27 10:38:33.025543 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] creating volume 'volume-66378b2c-0be3-45e8-8692-38c639675c8c' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:38:33.026300 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:33.084386 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:38:33.084791 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:38:33.088344 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5f061bb9-cb4a-44ad-83ef-ff042520bec5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-66378b2c-0be3-45e8-8692-38c639675c8c', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:33.092754 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (553ac07f-2930-489f-a7b8-0845b9cbaf14) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:38:33.104317 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Volume volume-66378b2c-0be3-45e8-8692-38c639675c8c (66378b2c-0be3-45e8-8692-38c639675c8c): created successfully Sep 27 10:38:33.109288 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (553ac07f-2930-489f-a7b8-0845b9cbaf14) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:38:33.114869 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Flow 'volume_create_manager' (9f2216d4-2ed1-4c19-b8c1-2dbc21f9e620) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:38:33.125360 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-d6b7a85e-c6ed-40a4-a290-74d191fd8016 tempest-VolumesBackupsV39Test-1623678555 None] Created volume successfully. Sep 27 10:38:34.168111 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-31e31e97-2904-49dc-91a6-6b64e4a0e932 tempest-VolumesBackupsV39Test-1623678555 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-27T10:38:34Z,data_timestamp=2022-09-27T10:38:34Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Backup-2120931482',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=ee3d0d21-fa79-4845-a3a1-4022629260cf,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='3de8116c00564abd82f93555f3f81993',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='64c73ef93e3b4d8da546f4ac18520560',volume_id=66378b2c-0be3-45e8-8692-38c639675c8c) Sep 27 10:38:34.168813 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.backup.rpcapi [None req-31e31e97-2904-49dc-91a6-6b64e4a0e932 tempest-VolumesBackupsV39Test-1623678555 None] continue_backup in rpcapi backup_id ee3d0d21-fa79-4845-a3a1-4022629260cf {{(pid=138473) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 27 10:38:34.254652 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-31e31e97-2904-49dc-91a6-6b64e4a0e932 tempest-VolumesBackupsV39Test-1623678555 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:38:35.007051 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-31e31e97-2904-49dc-91a6-6b64e4a0e932 tempest-VolumesBackupsV39Test-1623678555 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.752s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:38:35.008057 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-31e31e97-2904-49dc-91a6-6b64e4a0e932 tempest-VolumesBackupsV39Test-1623678555 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-66378b2c-0be3-45e8-8692-38c639675c8c', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '66378b2c-0be3-45e8-8692-38c639675c8c', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:38:35.025337 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-31e31e97-2904-49dc-91a6-6b64e4a0e932 tempest-VolumesBackupsV39Test-1623678555 None] Initialize volume connection completed successfully. Sep 27 10:38:37.260695 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-31e31e97-2904-49dc-91a6-6b64e4a0e932 tempest-VolumesBackupsV39Test-1623678555 None] Terminate volume connection completed successfully. Sep 27 10:38:37.278062 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-31e31e97-2904-49dc-91a6-6b64e4a0e932 tempest-VolumesBackupsV39Test-1623678555 None] Remove volume export completed successfully. Sep 27 10:38:37.659239 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=35,cinder:SELECT=56,cinder:INSERT=6,cinder:DELETE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:39.587366 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-550ec1e6-807c-4ef3-bbf0-83fc36f26e5c tempest-VolumesBackupsV39Test-1623678555 None] Acquiring lock "cinder-66378b2c-0be3-45e8-8692-38c639675c8c-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:38:39.597409 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-550ec1e6-807c-4ef3-bbf0-83fc36f26e5c tempest-VolumesBackupsV39Test-1623678555 None] Lock "cinder-66378b2c-0be3-45e8-8692-38c639675c8c-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:38:39.640180 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-550ec1e6-807c-4ef3-bbf0-83fc36f26e5c tempest-VolumesBackupsV39Test-1623678555 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:40.192271 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-550ec1e6-807c-4ef3-bbf0-83fc36f26e5c tempest-VolumesBackupsV39Test-1623678555 None] Volume volume-66378b2c-0be3-45e8-8692-38c639675c8c is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:38:40.208076 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-550ec1e6-807c-4ef3-bbf0-83fc36f26e5c tempest-VolumesBackupsV39Test-1623678555 None] deleting rbd volume volume-66378b2c-0be3-45e8-8692-38c639675c8c {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:38:40.404696 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=63,cinder:UPDATE=28,cinder:DELETE=3,cinder:INSERT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:40.421287 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-550ec1e6-807c-4ef3-bbf0-83fc36f26e5c tempest-VolumesBackupsV39Test-1623678555 None] Created reservations ['4d1a907b-6925-45ed-91d6-04b02c9da75a', '54cb0027-1d96-4c45-aaac-5dc449db2da1', '639e3acb-b637-4b04-b2f4-16d2303abdd2', 'ab08da59-4200-43cf-ac40-dab50e52b69d'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:38:40.485039 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-550ec1e6-807c-4ef3-bbf0-83fc36f26e5c tempest-VolumesBackupsV39Test-1623678555 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:40.513090 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-550ec1e6-807c-4ef3-bbf0-83fc36f26e5c tempest-VolumesBackupsV39Test-1623678555 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:38:40.539629 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-550ec1e6-807c-4ef3-bbf0-83fc36f26e5c tempest-VolumesBackupsV39Test-1623678555 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:38:40.545011 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-550ec1e6-807c-4ef3-bbf0-83fc36f26e5c tempest-VolumesBackupsV39Test-1623678555 None] Deleted volume successfully. Sep 27 10:38:40.560481 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-550ec1e6-807c-4ef3-bbf0-83fc36f26e5c tempest-VolumesBackupsV39Test-1623678555 None] Lock "cinder-66378b2c-0be3-45e8-8692-38c639675c8c-delete_volume" released by "delete_volume" :: held 0.963s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:38:57.653394 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=7,cinder:SELECT=4,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:17.661879 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:17.667900 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=17,cinder:INSERT=4,cinder:UPDATE=11 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:30.021042 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:39:30.021737 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:39:30.048532 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:39:30.075917 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:39:37.663451 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:37.670548 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:41.043905 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:39:41.064239 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Flow 'volume_create_manager' (970d1d1a-45db-4270-8153-6d6f7884ef78) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:39:41.070993 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0ca0f1f9-fb24-4d17-92b9-e2f8c98984a5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:41.086964 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0ca0f1f9-fb24-4d17-92b9-e2f8c98984a5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:39:41Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBackupRestore-volume-830643148',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cf5d9328-ecaa-4063-a5da-313e7db75d04,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='874d3b71bb7f44379f7efb4f68c7a14f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:39:41Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:39:41Z,use_quota=True,user_id='43ffa23d4c3843539e4be19528794e3c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:41.092191 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8475390a-8ecc-49e5-8cc3-575b51722755) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:41.096213 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8475390a-8ecc-49e5-8cc3-575b51722755) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:41.101158 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d51859f2-1320-4b2d-a28d-fb305d9570a5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:41.188700 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d51859f2-1320-4b2d-a28d-fb305d9570a5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'cf5d9328-ecaa-4063-a5da-313e7db75d04', 'volume_name': 'volume-cf5d9328-ecaa-4063-a5da-313e7db75d04', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:41.193439 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fa95c6ec-ba80-478c-a25d-f527f66f9372) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:41.197245 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fa95c6ec-ba80-478c-a25d-f527f66f9372) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:41.201780 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3bc050a0-a3d4-4305-bd88-7f28adc001b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:41.202873 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Volume cf5d9328-ecaa-4063-a5da-313e7db75d04: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-cf5d9328-ecaa-4063-a5da-313e7db75d04', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 10:39:41.203435 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Cloning cf5d9328-ecaa-4063-a5da-313e7db75d04 from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 10:39:41.204204 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:39:41.212443 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.008s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:39:41.224418 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 39, 41, 217192)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:39:41.224974 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:39:41.225496 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 10:39:41.231296 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.019s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:39:41.232682 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 10:39:41.241595 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 10, 39, 41, 237981)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 10:39:41.242102 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 10:39:41.242653 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 10:39:41.259792 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:39:41.321117 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] creating snapshot='volume-cf5d9328-ecaa-4063-a5da-313e7db75d04.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 10:39:42.385021 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-cf5d9328-ecaa-4063-a5da-313e7db75d04.clone_snap' to 'volume-cf5d9328-ecaa-4063-a5da-313e7db75d04' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 10:39:42.511495 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:39:42.512910 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:39:42.513281 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:39:42.530185 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 10:39:42.530960 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Marking volume cf5d9328-ecaa-4063-a5da-313e7db75d04 as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 10:39:42.540182 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to cf5d9328-ecaa-4063-a5da-313e7db75d04. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 10:39:42.540754 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Creating volume glance metadata for volume cf5d9328-ecaa-4063-a5da-313e7db75d04 backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 10:39:42.595951 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3bc050a0-a3d4-4305-bd88-7f28adc001b1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cf5d9328-ecaa-4063-a5da-313e7db75d04', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:42.600625 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bad9d2fb-b4ad-4529-a672-d2688f666287) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:42.615838 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Volume volume-cf5d9328-ecaa-4063-a5da-313e7db75d04 (cf5d9328-ecaa-4063-a5da-313e7db75d04): created successfully Sep 27 10:39:42.619635 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bad9d2fb-b4ad-4529-a672-d2688f666287) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:42.625318 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Flow 'volume_create_manager' (970d1d1a-45db-4270-8153-6d6f7884ef78) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:39:42.634953 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-a3c80071-bfa2-4475-924b-86e5dd64c8c7 tempest-TestVolumeBackupRestore-883386072 None] Created volume successfully. Sep 27 10:39:43.439633 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-4027ee98-99fa-4750-ba40-7ca8675088fb tempest-TestVolumeBackupRestore-883386072 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-27T10:39:43Z,data_timestamp=2022-09-27T10:39:43Z,deleted=False,deleted_at=None,display_description=None,display_name=None,encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=d9ccfd39-fc0e-4f10-9442-642b2528e01e,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='874d3b71bb7f44379f7efb4f68c7a14f',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='43ffa23d4c3843539e4be19528794e3c',volume_id=cf5d9328-ecaa-4063-a5da-313e7db75d04) Sep 27 10:39:43.440543 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.backup.rpcapi [None req-4027ee98-99fa-4750-ba40-7ca8675088fb tempest-TestVolumeBackupRestore-883386072 None] continue_backup in rpcapi backup_id d9ccfd39-fc0e-4f10-9442-642b2528e01e {{(pid=138473) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 27 10:39:43.528286 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-4027ee98-99fa-4750-ba40-7ca8675088fb tempest-TestVolumeBackupRestore-883386072 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:39:44.298644 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-4027ee98-99fa-4750-ba40-7ca8675088fb tempest-TestVolumeBackupRestore-883386072 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.770s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:39:44.299721 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-4027ee98-99fa-4750-ba40-7ca8675088fb tempest-TestVolumeBackupRestore-883386072 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-cf5d9328-ecaa-4063-a5da-313e7db75d04', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'cf5d9328-ecaa-4063-a5da-313e7db75d04', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:39:44.316877 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-4027ee98-99fa-4750-ba40-7ca8675088fb tempest-TestVolumeBackupRestore-883386072 None] Initialize volume connection completed successfully. Sep 27 10:39:48.407793 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-4027ee98-99fa-4750-ba40-7ca8675088fb tempest-TestVolumeBackupRestore-883386072 None] Terminate volume connection completed successfully. Sep 27 10:39:48.425190 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-4027ee98-99fa-4750-ba40-7ca8675088fb tempest-TestVolumeBackupRestore-883386072 None] Remove volume export completed successfully. Sep 27 10:39:48.975414 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:39:48.995150 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Flow 'volume_create_manager' (c0487ea0-350f-42c0-a8e4-e427e20468d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:39:49.001580 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ca1c0b4a-c214-4753-a9e0-541c2b89210f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:49.020751 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ca1c0b4a-c214-4753-a9e0-541c2b89210f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:39:49Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_d9ccfd39-fc0e-4f10-9442-642b2528e01e',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2b68bff0-c4ca-4aa3-b043-274fe56530cb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='874d3b71bb7f44379f7efb4f68c7a14f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:39:49Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:39:49Z,use_quota=True,user_id='43ffa23d4c3843539e4be19528794e3c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:49.025894 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (058f76aa-4d16-43e2-b114-6d452fc63fde) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:49.030112 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (058f76aa-4d16-43e2-b114-6d452fc63fde) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:49.035124 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (15cc3f47-19f1-49d5-91d6-f62351f7f085) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:49.039579 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (15cc3f47-19f1-49d5-91d6-f62351f7f085) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2b68bff0-c4ca-4aa3-b043-274fe56530cb', 'volume_name': 'volume-2b68bff0-c4ca-4aa3-b043-274fe56530cb', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:49.044280 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9e8b9e92-195a-4637-9d41-7a6a0d31ea0c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:49.048156 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9e8b9e92-195a-4637-9d41-7a6a0d31ea0c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:49.052587 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (309b7ebf-0497-487c-840b-d7cf2f2ab592) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:49.053668 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Volume 2b68bff0-c4ca-4aa3-b043-274fe56530cb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2b68bff0-c4ca-4aa3-b043-274fe56530cb', 'volume_size': 1} Sep 27 10:39:49.054074 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] creating volume 'volume-2b68bff0-c4ca-4aa3-b043-274fe56530cb' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:39:49.055220 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:39:49.126417 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:39:49.126715 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:39:49.130342 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (309b7ebf-0497-487c-840b-d7cf2f2ab592) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2b68bff0-c4ca-4aa3-b043-274fe56530cb', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:49.134694 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9fae2bc8-e318-4123-8acb-fd50a8fd2bcc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:39:49.146766 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Volume volume-2b68bff0-c4ca-4aa3-b043-274fe56530cb (2b68bff0-c4ca-4aa3-b043-274fe56530cb): created successfully Sep 27 10:39:49.150335 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9fae2bc8-e318-4123-8acb-fd50a8fd2bcc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:39:49.156013 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Flow 'volume_create_manager' (c0487ea0-350f-42c0-a8e4-e427e20468d9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:39:49.167373 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Created volume successfully. Sep 27 10:39:50.014863 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:39:50.772319 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.757s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:39:50.773148 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2b68bff0-c4ca-4aa3-b043-274fe56530cb', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '2b68bff0-c4ca-4aa3-b043-274fe56530cb', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:39:50.790933 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Initialize volume connection completed successfully. Sep 27 10:39:55.545941 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Terminate volume connection completed successfully. Sep 27 10:39:55.564011 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1d2e15d1-242f-4569-a327-0ebadcd99639 tempest-TestVolumeBackupRestore-883386072 None] Remove volume export completed successfully. Sep 27 10:39:59.978172 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c698d642-5e40-4c49-9e79-caf7834c34c8 req-07cc2fd1-4a30-4a57-be8e-eef66766fc9f tempest-TestVolumeBackupRestore-883386072 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:40:00.748697 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c698d642-5e40-4c49-9e79-caf7834c34c8 req-07cc2fd1-4a30-4a57-be8e-eef66766fc9f tempest-TestVolumeBackupRestore-883386072 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.770s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:40:00.750561 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-c698d642-5e40-4c49-9e79-caf7834c34c8 req-07cc2fd1-4a30-4a57-be8e-eef66766fc9f tempest-TestVolumeBackupRestore-883386072 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2b68bff0-c4ca-4aa3-b043-274fe56530cb', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '2b68bff0-c4ca-4aa3-b043-274fe56530cb', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:40:00.777343 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-c698d642-5e40-4c49-9e79-caf7834c34c8 req-07cc2fd1-4a30-4a57-be8e-eef66766fc9f tempest-TestVolumeBackupRestore-883386072 None] Connection info returned from driver {'name': 'volumes/volume-2b68bff0-c4ca-4aa3-b043-274fe56530cb', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '2b68bff0-c4ca-4aa3-b043-274fe56530cb', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '7a3a0a69-1f50-44e4-8174-df45f2990270'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:40:00.833323 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-c698d642-5e40-4c49-9e79-caf7834c34c8 req-07cc2fd1-4a30-4a57-be8e-eef66766fc9f tempest-TestVolumeBackupRestore-883386072 None] attachment_update completed successfully. Sep 27 10:40:17.657156 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=51,cinder:UPDATE=6,cinder:INSERT=8,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:17.658837 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=42,cinder:UPDATE=10,cinder:INSERT=4,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:30.022647 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:40:30.023627 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:40:30.050809 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:40:30.076937 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:40:47.666436 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:57.659613 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:59.549508 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-df49c12d-1bcb-4760-9b92-08303d91f097 req-a2672599-a886-4e20-8a85-2cb7b889f9c6 tempest-TestVolumeBackupRestore-883386072 None] Terminate volume connection completed successfully. Sep 27 10:40:59.550207 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-df49c12d-1bcb-4760-9b92-08303d91f097 req-a2672599-a886-4e20-8a85-2cb7b889f9c6 tempest-TestVolumeBackupRestore-883386072 None] Deleting attachment 7a3a0a69-1f50-44e4-8174-df45f2990270. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:41:00.904110 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5ec4f532-79ba-4d0a-ab90-96ae33b7ef86 tempest-TestVolumeBackupRestore-883386072 None] Acquiring lock "cinder-2b68bff0-c4ca-4aa3-b043-274fe56530cb-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:41:00.916914 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5ec4f532-79ba-4d0a-ab90-96ae33b7ef86 tempest-TestVolumeBackupRestore-883386072 None] Lock "cinder-2b68bff0-c4ca-4aa3-b043-274fe56530cb-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:41:00.952032 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ec4f532-79ba-4d0a-ab90-96ae33b7ef86 tempest-TestVolumeBackupRestore-883386072 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:41:01.370606 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ec4f532-79ba-4d0a-ab90-96ae33b7ef86 tempest-TestVolumeBackupRestore-883386072 None] Volume volume-2b68bff0-c4ca-4aa3-b043-274fe56530cb is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:41:01.385133 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ec4f532-79ba-4d0a-ab90-96ae33b7ef86 tempest-TestVolumeBackupRestore-883386072 None] deleting rbd volume volume-2b68bff0-c4ca-4aa3-b043-274fe56530cb {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:41:01.389807 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] Acquiring lock "cinder-cf5d9328-ecaa-4063-a5da-313e7db75d04-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:41:01.399460 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] Lock "cinder-cf5d9328-ecaa-4063-a5da-313e7db75d04-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:41:01.436941 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:41:02.419062 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] deleting rbd volume volume-cf5d9328-ecaa-4063-a5da-313e7db75d04 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:41:02.472474 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-5ec4f532-79ba-4d0a-ab90-96ae33b7ef86 tempest-TestVolumeBackupRestore-883386072 None] Created reservations ['23b714ca-bee3-4e27-8f70-100da038f1be', '4e300489-3cbc-4458-9974-f05ecb41d388', '23a96ab2-a68a-43aa-bb0a-ea5d50002763', '245bcd6f-d747-43ae-b730-363b13b491dd'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:41:02.524949 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ec4f532-79ba-4d0a-ab90-96ae33b7ef86 tempest-TestVolumeBackupRestore-883386072 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:41:02.550958 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-5ec4f532-79ba-4d0a-ab90-96ae33b7ef86 tempest-TestVolumeBackupRestore-883386072 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:41:02.578169 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-5ec4f532-79ba-4d0a-ab90-96ae33b7ef86 tempest-TestVolumeBackupRestore-883386072 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:41:02.584308 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-5ec4f532-79ba-4d0a-ab90-96ae33b7ef86 tempest-TestVolumeBackupRestore-883386072 None] Deleted volume successfully. Sep 27 10:41:02.590419 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 10:41:02.622728 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] deleting parent snapshot volume-cf5d9328-ecaa-4063-a5da-313e7db75d04.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 10:41:03.461102 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-5ec4f532-79ba-4d0a-ab90-96ae33b7ef86 tempest-TestVolumeBackupRestore-883386072 None] Lock "cinder-2b68bff0-c4ca-4aa3-b043-274fe56530cb-delete_volume" released by "delete_volume" :: held 2.544s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:41:03.495843 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] Created reservations ['5fce64c8-b197-4a05-b59d-a9b40c3168d9', '3e3f40bb-a404-4fba-ace9-8561f335f2d0', '2664505e-f86a-49b3-8d95-ce73ffc0d369', 'e24e9247-2632-47f0-b0d9-be4ce42b6ba6'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:41:03.551081 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:41:03.583610 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:41:03.611632 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:41:03.615984 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] Deleted volume successfully. Sep 27 10:41:03.630505 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-1d150c64-93cb-4357-9274-a781c7af6695 tempest-TestVolumeBackupRestore-883386072 None] Lock "cinder-cf5d9328-ecaa-4063-a5da-313e7db75d04-delete_volume" released by "delete_volume" :: held 2.231s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:41:27.661426 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=28,cinder:UPDATE=22,cinder:INSERT=4,cinder:DELETE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:30.022415 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:41:30.023106 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:41:30.047626 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:41:30.071868 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:41:37.657252 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=36,cinder:INSERT=4,cinder:UPDATE=14 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:57.660557 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:07.656146 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:30.027823 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:42:30.028586 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:42:30.057512 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:42:30.087058 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:42:37.648259 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:37.652421 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:57.650669 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:17.646009 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:17.650152 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:30.032427 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:43:30.033535 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:43:30.060301 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:43:30.083728 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:43:47.650021 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:57.642851 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:27.636410 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:30.029479 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:44:30.030258 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:44:30.057688 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:44:30.084777 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:44:37.640470 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:47.642204 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:45:07.642414 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:45:27.638021 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:45:30.030893 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:45:30.031368 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:45:30.057978 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:45:30.085321 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:45:47.632830 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:07.640058 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:07.645042 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:27.652907 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:30.030099 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:46:30.030740 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:46:30.065150 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:46:30.088936 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:46:47.644491 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:47:17.649228 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:47:17.658194 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:47:30.034974 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:47:30.036089 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:47:30.071746 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:47:30.096136 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:47:47.652800 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:47:57.639166 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:48:27.636112 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:48:27.639204 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:48:30.031413 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:48:30.032124 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:48:30.058453 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:48:30.085052 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:49:07.638558 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:07.640065 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:30.031188 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:49:30.031942 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:49:30.060793 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:49:30.087351 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:49:47.630873 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:47.632403 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:17.632956 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:27.641382 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:30.032323 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:50:30.033181 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:50:30.068560 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:50:30.095181 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:50:47.646881 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:01.387745 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:01.408268 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:51:01.428741 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Flow 'volume_create_manager' (286ab2b8-8c79-4fb7-94fc-fbe50f91f4fc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:51:01.434890 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ef7e1ba8-0c5f-4f88-8967-ce9eb396891a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:51:01.454831 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ef7e1ba8-0c5f-4f88-8967-ce9eb396891a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:51:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerActionsTestJSON-volume-1563546059',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c5d7e76d-768e-416c-a698-6f718f9340c4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='32845fb8bd064f55bb99d01e5aeff6f3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:51:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:51:01Z,use_quota=True,user_id='396fe9ce0f7542c382b01d1f5779c13f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:51:01.461912 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (363f7dcc-8571-4155-8efd-11bc79b0eebd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:51:01.468000 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (363f7dcc-8571-4155-8efd-11bc79b0eebd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:51:01.475307 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (96cfa4c9-ec85-4f35-a7f3-81360f8ee7da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:51:01.479240 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (96cfa4c9-ec85-4f35-a7f3-81360f8ee7da) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c5d7e76d-768e-416c-a698-6f718f9340c4', 'volume_name': 'volume-c5d7e76d-768e-416c-a698-6f718f9340c4', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:51:01.484072 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7e47cb5b-4c48-4872-8cb1-f0e9faeb3e16) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:51:01.487800 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7e47cb5b-4c48-4872-8cb1-f0e9faeb3e16) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:51:01.492398 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (90c430f5-6fa5-4b20-84f5-29fad2def43f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:51:01.493852 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Volume c5d7e76d-768e-416c-a698-6f718f9340c4: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c5d7e76d-768e-416c-a698-6f718f9340c4', 'volume_size': 1} Sep 27 10:51:01.494117 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] creating volume 'volume-c5d7e76d-768e-416c-a698-6f718f9340c4' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:51:01.494912 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:51:01.574723 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:51:01.575205 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:51:01.581510 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (90c430f5-6fa5-4b20-84f5-29fad2def43f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c5d7e76d-768e-416c-a698-6f718f9340c4', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:51:01.587284 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9be07aa5-b118-4786-b039-1a28fc518363) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:51:01.600783 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Volume volume-c5d7e76d-768e-416c-a698-6f718f9340c4 (c5d7e76d-768e-416c-a698-6f718f9340c4): created successfully Sep 27 10:51:01.605469 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9be07aa5-b118-4786-b039-1a28fc518363) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:51:01.612738 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Flow 'volume_create_manager' (286ab2b8-8c79-4fb7-94fc-fbe50f91f4fc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:51:01.627422 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-035e4276-1647-4870-9d9b-cb712d9d1de8 tempest-ServerActionsTestJSON-429271553 None] Created volume successfully. Sep 27 10:51:03.097131 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-b7e7255e-a594-4b33-8042-1027c4a0e712 req-f7668fb5-36ad-4c85-a69f-243b219d191e tempest-ServerActionsTestJSON-429271553 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:51:03.855002 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-b7e7255e-a594-4b33-8042-1027c4a0e712 req-f7668fb5-36ad-4c85-a69f-243b219d191e tempest-ServerActionsTestJSON-429271553 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.757s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:51:03.856125 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-b7e7255e-a594-4b33-8042-1027c4a0e712 req-f7668fb5-36ad-4c85-a69f-243b219d191e tempest-ServerActionsTestJSON-429271553 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-c5d7e76d-768e-416c-a698-6f718f9340c4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'c5d7e76d-768e-416c-a698-6f718f9340c4', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:51:03.890093 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-b7e7255e-a594-4b33-8042-1027c4a0e712 req-f7668fb5-36ad-4c85-a69f-243b219d191e tempest-ServerActionsTestJSON-429271553 None] Connection info returned from driver {'name': 'volumes/volume-c5d7e76d-768e-416c-a698-6f718f9340c4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'c5d7e76d-768e-416c-a698-6f718f9340c4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '8a8c5092-7480-4cc1-b63f-5642fad254a9'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:51:03.941611 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-b7e7255e-a594-4b33-8042-1027c4a0e712 req-f7668fb5-36ad-4c85-a69f-243b219d191e tempest-ServerActionsTestJSON-429271553 None] attachment_update completed successfully. Sep 27 10:51:20.893553 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-9238330c-7598-4fd2-9f18-1a9a94c715cd req-468a384e-e7e3-4fbb-943d-cb518a93db6a tempest-ServerActionsTestJSON-429271553 None] Terminate volume connection completed successfully. Sep 27 10:51:20.894184 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-9238330c-7598-4fd2-9f18-1a9a94c715cd req-468a384e-e7e3-4fbb-943d-cb518a93db6a tempest-ServerActionsTestJSON-429271553 None] Deleting attachment 8a8c5092-7480-4cc1-b63f-5642fad254a9. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:51:22.982904 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-9238330c-7598-4fd2-9f18-1a9a94c715cd req-931472b8-ab39-458c-8628-dccfc97d6ea1 tempest-ServerActionsTestJSON-429271553 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:51:23.741967 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-9238330c-7598-4fd2-9f18-1a9a94c715cd req-931472b8-ab39-458c-8628-dccfc97d6ea1 tempest-ServerActionsTestJSON-429271553 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.759s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:51:23.742844 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-9238330c-7598-4fd2-9f18-1a9a94c715cd req-931472b8-ab39-458c-8628-dccfc97d6ea1 tempest-ServerActionsTestJSON-429271553 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-c5d7e76d-768e-416c-a698-6f718f9340c4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'c5d7e76d-768e-416c-a698-6f718f9340c4', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:51:23.770318 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-9238330c-7598-4fd2-9f18-1a9a94c715cd req-931472b8-ab39-458c-8628-dccfc97d6ea1 tempest-ServerActionsTestJSON-429271553 None] Connection info returned from driver {'name': 'volumes/volume-c5d7e76d-768e-416c-a698-6f718f9340c4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'c5d7e76d-768e-416c-a698-6f718f9340c4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '56376971-53c9-4637-a9d5-e96c6394c0c9'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:51:23.822248 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-9238330c-7598-4fd2-9f18-1a9a94c715cd req-931472b8-ab39-458c-8628-dccfc97d6ea1 tempest-ServerActionsTestJSON-429271553 None] attachment_update completed successfully. Sep 27 10:51:27.666340 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=29,cinder:UPDATE=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:30.030186 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:51:30.031178 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:51:30.061541 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:51:30.098003 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:51:32.360519 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=6,cinder:SELECT=34,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:32.373144 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-d1f52421-44c1-4a35-9d89-eb2219269a1e req-16fbab9b-70f5-4682-9230-12c8b2a2faa2 tempest-ServerActionsTestJSON-429271553 None] Terminate volume connection completed successfully. Sep 27 10:51:32.374072 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-d1f52421-44c1-4a35-9d89-eb2219269a1e req-16fbab9b-70f5-4682-9230-12c8b2a2faa2 tempest-ServerActionsTestJSON-429271553 None] Deleting attachment 56376971-53c9-4637-a9d5-e96c6394c0c9. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:51:34.708411 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-d1f52421-44c1-4a35-9d89-eb2219269a1e req-642598da-a800-4bdf-a1ee-9510926844d5 tempest-ServerActionsTestJSON-429271553 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:51:35.437479 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-d1f52421-44c1-4a35-9d89-eb2219269a1e req-642598da-a800-4bdf-a1ee-9510926844d5 tempest-ServerActionsTestJSON-429271553 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.729s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:51:35.438348 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-d1f52421-44c1-4a35-9d89-eb2219269a1e req-642598da-a800-4bdf-a1ee-9510926844d5 tempest-ServerActionsTestJSON-429271553 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-c5d7e76d-768e-416c-a698-6f718f9340c4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'c5d7e76d-768e-416c-a698-6f718f9340c4', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:51:35.472310 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-d1f52421-44c1-4a35-9d89-eb2219269a1e req-642598da-a800-4bdf-a1ee-9510926844d5 tempest-ServerActionsTestJSON-429271553 None] Connection info returned from driver {'name': 'volumes/volume-c5d7e76d-768e-416c-a698-6f718f9340c4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'c5d7e76d-768e-416c-a698-6f718f9340c4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '245356d1-2424-474f-95a0-c0ef75bf0437'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:51:35.525203 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-d1f52421-44c1-4a35-9d89-eb2219269a1e req-642598da-a800-4bdf-a1ee-9510926844d5 tempest-ServerActionsTestJSON-429271553 None] attachment_update completed successfully. Sep 27 10:51:41.566571 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-c7c64898-1e24-4169-8f17-82fdc34efbb4 req-b8b567ee-5e21-4712-9086-47296476e044 tempest-ServerActionsTestJSON-429271553 None] Terminate volume connection completed successfully. Sep 27 10:51:41.567255 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-c7c64898-1e24-4169-8f17-82fdc34efbb4 req-b8b567ee-5e21-4712-9086-47296476e044 tempest-ServerActionsTestJSON-429271553 None] Deleting attachment 245356d1-2424-474f-95a0-c0ef75bf0437. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:51:42.685416 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] Acquiring lock "cinder-c5d7e76d-768e-416c-a698-6f718f9340c4-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:51:42.695521 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] Lock "cinder-c5d7e76d-768e-416c-a698-6f718f9340c4-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:51:42.729469 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:51:42.769211 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:51:42.769587 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] Volume volume-c5d7e76d-768e-416c-a698-6f718f9340c4 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:51:42.774330 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] deleting rbd volume volume-c5d7e76d-768e-416c-a698-6f718f9340c4 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:51:42.977118 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] Created reservations ['7a83889f-ee09-422c-9a87-2e9c485ef688', '9d086942-6bf9-4355-9c5a-efe932728087', '24d3a9ed-19ae-41fc-80da-a145c0cbb132', '5104c32c-20d5-49c1-aad6-afd747046369'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:51:43.027584 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:51:43.053461 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:51:43.080375 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:51:43.085468 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] Deleted volume successfully. Sep 27 10:51:43.099490 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-63863079-3c06-4637-93e4-b58cf6e990b0 tempest-ServerActionsTestJSON-429271553 None] Lock "cinder-c5d7e76d-768e-416c-a698-6f718f9340c4-delete_volume" released by "delete_volume" :: held 0.404s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:51:53.090423 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=25,cinder:UPDATE=11,cinder:INSERT=2,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:07.652451 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=36,cinder:UPDATE=10,cinder:INSERT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:27.657290 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:30.033599 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:52:30.034311 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:52:30.060456 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:52:30.094414 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:52:37.654354 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:47.660767 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:17.662625 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:27.658276 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:30.029786 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:53:30.031110 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:53:30.102040 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:53:30.130410 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:53:30.155695 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:53:30.176342 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Flow 'volume_create_manager' (a81da2f6-8b27-4e84-81b2-b016fcd7315c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:53:30.182628 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1a5734a0-eedf-4419-8d6e-0b15c471dc31) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:53:30.199642 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1a5734a0-eedf-4419-8d6e-0b15c471dc31) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:53:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-209825797',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ff248593-4867-4300-83fa-11e79e5e1405,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='714fa95d9be140018a54ece37d204047',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:53:30Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:53:30Z,use_quota=True,user_id='906ca88c8f544940b06705711ae7f1af',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:53:30.204633 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (698c14de-d1f7-49e7-afbd-beed4b0ff772) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:53:30.208524 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (698c14de-d1f7-49e7-afbd-beed4b0ff772) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:53:30.213595 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (161cf2e8-5aa3-4c33-8e5e-a654ac9e077f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:53:30.217551 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (161cf2e8-5aa3-4c33-8e5e-a654ac9e077f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ff248593-4867-4300-83fa-11e79e5e1405', 'volume_name': 'volume-ff248593-4867-4300-83fa-11e79e5e1405', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:53:30.222245 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (651ece66-eb6b-4fb5-8e46-51e39cd4340b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:53:30.225956 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (651ece66-eb6b-4fb5-8e46-51e39cd4340b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:53:30.230497 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (48de72a0-a382-4707-8397-d74ac8995bdd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:53:30.231533 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Volume ff248593-4867-4300-83fa-11e79e5e1405: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ff248593-4867-4300-83fa-11e79e5e1405', 'volume_size': 1} Sep 27 10:53:30.231895 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] creating volume 'volume-ff248593-4867-4300-83fa-11e79e5e1405' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:53:30.232676 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:53:30.324677 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:53:30.325084 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:53:30.328724 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (48de72a0-a382-4707-8397-d74ac8995bdd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ff248593-4867-4300-83fa-11e79e5e1405', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:53:30.333133 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4071790f-e09c-4b22-a02f-d353a0666fae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:53:30.345104 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Volume volume-ff248593-4867-4300-83fa-11e79e5e1405 (ff248593-4867-4300-83fa-11e79e5e1405): created successfully Sep 27 10:53:30.348869 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4071790f-e09c-4b22-a02f-d353a0666fae) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:53:30.354645 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Flow 'volume_create_manager' (a81da2f6-8b27-4e84-81b2-b016fcd7315c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:53:30.363747 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-b682cee7-71d2-45b2-a1f3-cd5896100d23 tempest-TestMinimumBasicScenario-1586443371 None] Created volume successfully. Sep 27 10:53:31.859998 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-826e7898-c713-46c0-b0c5-d273a4a671a5 req-10231851-bdef-4cbc-9613-77c938be206b tempest-TestMinimumBasicScenario-1586443371 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:53:32.996255 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-826e7898-c713-46c0-b0c5-d273a4a671a5 req-10231851-bdef-4cbc-9613-77c938be206b tempest-TestMinimumBasicScenario-1586443371 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.136s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:53:32.997076 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-826e7898-c713-46c0-b0c5-d273a4a671a5 req-10231851-bdef-4cbc-9613-77c938be206b tempest-TestMinimumBasicScenario-1586443371 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-ff248593-4867-4300-83fa-11e79e5e1405', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': 'ff248593-4867-4300-83fa-11e79e5e1405', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:53:33.025584 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-826e7898-c713-46c0-b0c5-d273a4a671a5 req-10231851-bdef-4cbc-9613-77c938be206b tempest-TestMinimumBasicScenario-1586443371 None] Connection info returned from driver {'name': 'volumes/volume-ff248593-4867-4300-83fa-11e79e5e1405', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'ff248593-4867-4300-83fa-11e79e5e1405', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '2da146a7-d5e5-4868-87b4-56fb5aa26bdb'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:53:33.077611 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-826e7898-c713-46c0-b0c5-d273a4a671a5 req-10231851-bdef-4cbc-9613-77c938be206b tempest-TestMinimumBasicScenario-1586443371 None] attachment_update completed successfully. Sep 27 10:53:35.215253 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-84a51cdb-45c2-4014-b5d5-03b1d24705b9 tempest-TestMinimumBasicScenario-1586443371 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:53:36.375035 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-84a51cdb-45c2-4014-b5d5-03b1d24705b9 tempest-TestMinimumBasicScenario-1586443371 None] Create snapshot completed successfully Sep 27 10:53:57.659984 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=23,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:54:07.655202 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=29,cinder:UPDATE=7,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:54:30.034176 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:54:30.035001 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:54:30.067061 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:54:30.097489 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:54:37.648207 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:54:47.645792 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:55:17.638508 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:55:27.636082 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:55:30.033785 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:55:30.035250 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:55:30.065204 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:55:30.089391 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:55:57.632846 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:56:03.795461 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-cdc656ff-adf4-4435-b51b-6fe8205543e3 tempest-TestMinimumBasicScenario-1586443371 None] Acquiring lock "cinder-2dc9bc0f-9c7a-4778-93d8-b80d01754729-delete_snapshot" by "delete_snapshot" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:56:03.807686 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-cdc656ff-adf4-4435-b51b-6fe8205543e3 tempest-TestMinimumBasicScenario-1586443371 None] Lock "cinder-2dc9bc0f-9c7a-4778-93d8-b80d01754729-delete_snapshot" acquired by "delete_snapshot" :: waited 0.013s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:56:03.809542 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-cdc656ff-adf4-4435-b51b-6fe8205543e3 tempest-TestMinimumBasicScenario-1586443371 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:56:07.632845 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:56:30.035421 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:56:30.036618 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:56:30.071113 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:56:30.093525 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:56:33.922208 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:56:33.971938 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-cdc656ff-adf4-4435-b51b-6fe8205543e3 tempest-TestMinimumBasicScenario-1586443371 None] Created reservations ['b43a626a-6e5c-4e66-98ff-2ae04842a71e', '9dc5c1c2-19be-4a3c-a810-13fd30a41256', 'ff6093b6-5377-4539-a6b8-648757b65f5a', '2e162944-3274-484c-900d-a3139520e06c'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:56:34.010113 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-cdc656ff-adf4-4435-b51b-6fe8205543e3 tempest-TestMinimumBasicScenario-1586443371 None] Delete snapshot completed successfully. Sep 27 10:56:34.014754 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-cdc656ff-adf4-4435-b51b-6fe8205543e3 tempest-TestMinimumBasicScenario-1586443371 None] Lock "cinder-2dc9bc0f-9c7a-4778-93d8-b80d01754729-delete_snapshot" released by "delete_snapshot" :: held 30.207s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:56:47.625246 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=12,cinder:UPDATE=10,cinder:INSERT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:57:07.625540 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=17,cinder:INSERT=2,cinder:UPDATE=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:57:27.629111 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:57:27.629785 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:57:30.039951 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:57:30.041687 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:57:30.077971 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:57:30.103124 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:57:57.636117 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:57:57.641396 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:16.462582 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f req-42a47fd0-fdd7-4044-8a79-b1db1fa97508 tempest-TestMinimumBasicScenario-1586443371 None] Terminate volume connection completed successfully. Sep 27 10:58:16.463365 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f req-42a47fd0-fdd7-4044-8a79-b1db1fa97508 tempest-TestMinimumBasicScenario-1586443371 None] Deleting attachment 2da146a7-d5e5-4868-87b4-56fb5aa26bdb. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 10:58:16.975220 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] Acquiring lock "cinder-ff248593-4867-4300-83fa-11e79e5e1405-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 10:58:16.985223 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] Lock "cinder-ff248593-4867-4300-83fa-11e79e5e1405-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 10:58:17.019612 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:58:17.056787 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 10:58:17.057411 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] Volume volume-ff248593-4867-4300-83fa-11e79e5e1405 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 10:58:17.064713 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] deleting rbd volume volume-ff248593-4867-4300-83fa-11e79e5e1405 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 10:58:17.264877 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] Created reservations ['f27152f7-1798-4271-bd97-d2b77310976c', 'e76a60a3-df7b-4999-a41a-d38e066e3456', '9dc1ad61-93a6-4b31-a1a0-207db81fd8d7', '999fbf09-c771-4cc9-b514-a5344129d1a7'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 10:58:17.315602 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:58:17.346212 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:58:17.370596 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:58:17.374878 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] Deleted volume successfully. Sep 27 10:58:17.388736 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-8fc53198-0240-4bc1-b65d-23701b227035 tempest-TestMinimumBasicScenario-1586443371 None] Lock "cinder-ff248593-4867-4300-83fa-11e79e5e1405-delete_volume" released by "delete_volume" :: held 0.403s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 10:58:27.636425 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=23,cinder:INSERT=2,cinder:UPDATE=8 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:27.644031 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=11,cinder:SELECT=14,cinder:INSERT=2,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:30.038907 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:58:30.039613 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:58:30.074628 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:58:30.106979 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:58:49.902282 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 10:58:49.922662 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Flow 'volume_create_manager' (2934044a-7601-42b1-9656-0c926cf46584) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:58:49.928898 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ed3e93e8-8fd2-43c8-b78d-5e39b708dd1d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:58:49.945760 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ed3e93e8-8fd2-43c8-b78d-5e39b708dd1d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T10:58:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1299032143',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='714fa95d9be140018a54ece37d204047',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T10:58:50Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T10:58:50Z,use_quota=True,user_id='906ca88c8f544940b06705711ae7f1af',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:58:49.951047 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b24d9917-a8e9-476d-a515-21238447c2ad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:58:49.954986 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b24d9917-a8e9-476d-a515-21238447c2ad) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:58:49.959784 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4d4657e9-d131-4005-977f-eeeee96643bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:58:49.963668 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4d4657e9-d131-4005-977f-eeeee96643bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97', 'volume_name': 'volume-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:58:49.968981 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ddb0d083-70a4-4f75-88ea-a148c585b02b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:58:49.972712 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ddb0d083-70a4-4f75-88ea-a148c585b02b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:58:49.977135 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e6687184-1336-47f7-8ba5-ffa5785f9b96) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:58:49.978178 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Volume 022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97', 'volume_size': 1} Sep 27 10:58:49.978578 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] creating volume 'volume-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97' {{(pid=138473) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 27 10:58:49.979341 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:58:50.128086 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 10:58:50.128539 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 10:58:50.132127 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e6687184-1336-47f7-8ba5-ffa5785f9b96) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97', 'volume_size': 1}' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:58:50.136448 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4aa5358b-9ab9-41bb-8df1-e113714aef38) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 10:58:50.147307 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Volume volume-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97 (022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97): created successfully Sep 27 10:58:50.150625 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4aa5358b-9ab9-41bb-8df1-e113714aef38) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 10:58:50.155780 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Flow 'volume_create_manager' (2934044a-7601-42b1-9656-0c926cf46584) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 10:58:50.165675 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-6e95276a-a024-4143-a909-86a66ece9a1c tempest-TestMinimumBasicScenario-1586443371 None] Created volume successfully. Sep 27 10:58:51.551803 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-7968931f-a63a-4376-95c8-fd49b159f3e4 req-7d6db5d5-e37e-46f9-9505-8035f0523825 tempest-TestMinimumBasicScenario-1586443371 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:58:52.393098 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-7968931f-a63a-4376-95c8-fd49b159f3e4 req-7d6db5d5-e37e-46f9-9505-8035f0523825 tempest-TestMinimumBasicScenario-1586443371 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.840s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:58:52.394328 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-7968931f-a63a-4376-95c8-fd49b159f3e4 req-7d6db5d5-e37e-46f9-9505-8035f0523825 tempest-TestMinimumBasicScenario-1586443371 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 10:58:52.433453 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-7968931f-a63a-4376-95c8-fd49b159f3e4 req-7d6db5d5-e37e-46f9-9505-8035f0523825 tempest-TestMinimumBasicScenario-1586443371 None] Connection info returned from driver {'name': 'volumes/volume-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'f7d00c25-596a-41d3-b043-c286ae565c70'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 10:58:52.490574 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-7968931f-a63a-4376-95c8-fd49b159f3e4 req-7d6db5d5-e37e-46f9-9505-8035f0523825 tempest-TestMinimumBasicScenario-1586443371 None] attachment_update completed successfully. Sep 27 10:59:07.634322 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=7,cinder:SELECT=22 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:59:17.632605 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=20,cinder:UPDATE=4,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:59:30.041479 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 10:59:30.043165 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:59:30.075860 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 10:59:30.117087 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 10:59:47.629989 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:59:57.627116 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:09.262893 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-49c79232-841f-4850-b1ce-61d51110a069 req-1675cc90-a898-4730-9fe8-99a9cccdccb7 tempest-TestMinimumBasicScenario-1586443371 None] Terminate volume connection completed successfully. Sep 27 11:00:09.263493 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-49c79232-841f-4850-b1ce-61d51110a069 req-1675cc90-a898-4730-9fe8-99a9cccdccb7 tempest-TestMinimumBasicScenario-1586443371 None] Deleting attachment f7d00c25-596a-41d3-b043-c286ae565c70. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 11:00:09.892229 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] Acquiring lock "cinder-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 11:00:09.903551 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] Lock "cinder-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 11:00:09.938680 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:00:09.977304 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 11:00:09.978300 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] Volume volume-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97 is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 11:00:09.984321 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] deleting rbd volume volume-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97 {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 11:00:10.171814 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] Created reservations ['a57ac70d-9ee6-4a5a-9707-dbe09e171477', '2dbd3e58-e64b-4d5c-ba18-3a9345f7e175', 'fae8f5eb-a1cd-4d89-9fe9-f8957d0da909', '12387eee-36dd-4111-9ab6-27819a4dc74b'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 11:00:10.228324 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:00:10.254012 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:00:10.278654 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:00:10.284142 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] Deleted volume successfully. Sep 27 11:00:10.299660 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-92fcd1ec-e30c-4690-9893-9e7c375b9b99 tempest-TestMinimumBasicScenario-1586443371 None] Lock "cinder-022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97-delete_volume" released by "delete_volume" :: held 0.396s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 11:00:27.623728 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=22,cinder:UPDATE=11,cinder:INSERT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:30.042765 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:00:30.043580 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:00:30.079447 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:00:30.133930 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:00:37.622034 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=18,cinder:UPDATE=9,cinder:INSERT=2,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:01:07.620555 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:01:17.621931 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:01:30.040408 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:01:30.041172 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:01:30.067703 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:01:30.094774 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:01:47.619053 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:01:57.619571 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:17.623153 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:30.040280 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:02:30.041063 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:02:30.067542 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:02:30.094343 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:02:37.620911 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:57.616588 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:17.614885 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:30.040558 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:03:30.041686 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:03:30.073459 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:03:30.097620 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:03:37.622683 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:57.622221 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:17.614473 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=9,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:30.042790 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:04:30.043503 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:04:30.067411 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:04:30.091743 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:04:37.616801 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:47.624911 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:05:17.618562 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:05:27.617485 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=3,cinder:SELECT=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:05:30.047470 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:05:30.048555 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:05:30.082034 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:05:30.106586 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:05:57.613654 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:07.620604 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:27.622770 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:30.052168 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:06:30.053061 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:06:30.102471 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:06:30.145804 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:06:37.616159 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:07:07.608998 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:07:17.614102 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:07:27.616394 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:07:30.049176 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:07:30.049889 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:07:30.077211 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:07:30.102034 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:07:57.613676 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:08:07.619386 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:08:30.048124 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:08:30.049109 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:08:30.078556 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:08:30.104074 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:08:37.615442 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:08:47.618542 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:17.612051 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:27.615088 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:30.049469 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:09:30.050239 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:09:30.080362 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:09:30.111997 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:09:37.628438 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:57.622758 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:17.621111 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:30.047911 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:10:30.049200 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:10:30.083954 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:10:30.111693 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:10:37.620899 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:57.623755 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:07.637092 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:27.640651 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:30.048156 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:11:30.049430 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:11:30.079037 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:11:30.111168 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:11:47.631278 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:12:07.632065 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:12:07.636862 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:12:30.049733 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:12:30.050866 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:12:30.083409 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:12:30.107149 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:12:37.640124 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:12:37.647337 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:17.633403 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=4,cinder:SELECT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:17.635011 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:30.054336 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:13:30.055400 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:13:30.090818 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:13:30.115795 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:13:57.629625 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=9,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:57.631176 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:14:30.052537 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:14:30.053225 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:14:30.077559 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:14:30.102044 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:14:37.619669 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:14:37.621349 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:14:57.630602 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:15:07.631397 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:15:30.057331 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:15:30.058755 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:15:30.084301 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:15:30.108358 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:15:37.623419 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:15:47.621712 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:15:48.151570 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=138473) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Sep 27 11:15:48.170560 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Flow 'volume_create_manager' (9b793d8e-8cd7-4a66-97d9-fcb7b6160aa6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 11:15:48.176806 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2bae56c6-1782-4879-aff5-deac748010da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 11:15:48.194174 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2bae56c6-1782-4879-aff5-deac748010da) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-27T11:15:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-514198787',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=570e326c-0f5d-44bd-a289-2e89c4df917f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='37db46b0eb6a4c7d938085b26d1968a7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-27T11:15:48Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-27T11:15:48Z,use_quota=True,user_id='90cea8d7a5494c45ba299df59797633f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e6016daf-4ebe-4600-800c-f0685eb2234c),volume_type_id=e6016daf-4ebe-4600-800c-f0685eb2234c)' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 11:15:48.199259 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (42c4f519-623b-4b04-9caf-4671a2ef1672) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 11:15:48.203348 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (42c4f519-623b-4b04-9caf-4671a2ef1672) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 11:15:48.208185 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d6a168af-2f60-4805-9925-c638a72ed9db) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 11:15:48.299039 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d6a168af-2f60-4805-9925-c638a72ed9db) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '570e326c-0f5d-44bd-a289-2e89c4df917f', 'volume_name': 'volume-570e326c-0f5d-44bd-a289-2e89c4df917f', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 11:15:48.303948 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f03ab00c-a448-4a83-bc5c-cadb260a5e9e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 11:15:48.307847 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f03ab00c-a448-4a83-bc5c-cadb260a5e9e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 11:15:48.312371 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (09b4d6bc-dd1b-4df6-aed5-d77776eff403) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 11:15:48.313556 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Volume 570e326c-0f5d-44bd-a289-2e89c4df917f: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-570e326c-0f5d-44bd-a289-2e89c4df917f', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 27 11:15:48.314135 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Cloning 570e326c-0f5d-44bd-a289-2e89c4df917f from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 at location (None, None). {{(pid=138473) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Sep 27 11:15:48.314901 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Acquiring lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" by "_prepare_image_cache_entry" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 11:15:48.324078 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 11:15:48.335462 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 11, 15, 48, 329094)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 11:15:48.335960 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 11:15:48.336427 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Found cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Sep 27 11:15:48.347196 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Lock "cinder-c1623d3a-c1bb-4649-9910-5deadb8a1ff6" released by "_prepare_image_cache_entry" :: held 0.023s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 11:15:48.348202 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Attempting to retrieve cache entry for image = c1623d3a-c1bb-4649-9910-5deadb8a1ff6 on host ncdv-master-1@ceph#ceph. {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Sep 27 11:15:48.354973 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Found image-volume cache entry: {'id': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'volume_id': '78171a54-b27c-4c21-b4c6-bf0ecca1bcd1', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29), 'last_used': datetime.datetime(2022, 9, 27, 11, 15, 48, 352382)}. {{(pid=138473) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 27 11:15:48.355392 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.image.cache [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Image-volume cache entry image_update_at = 2022-09-27 09:47:29+00:00, requested image updated_at = 2022-09-27 09:47:29+00:00. {{(pid=138473) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 27 11:15:48.355809 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Creating from source image-volume 78171a54-b27c-4c21-b4c6-bf0ecca1bcd1 {{(pid=138473) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Sep 27 11:15:48.370442 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:15:48.442762 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] creating snapshot='volume-570e326c-0f5d-44bd-a289-2e89c4df917f.clone_snap' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 27 11:15:49.434329 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] cloning 'volume-78171a54-b27c-4c21-b4c6-bf0ecca1bcd1@volume-570e326c-0f5d-44bd-a289-2e89c4df917f.clone_snap' to 'volume-570e326c-0f5d-44bd-a289-2e89c4df917f' {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 27 11:15:49.613101 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Volume volume-ff80fef3-6188-4603-afd2-55aa0b605e0c.deleted is not a clone. {{(pid=138473) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 27 11:15:49.614608 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 27 11:15:49.614974 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=138473) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 27 11:15:49.631889 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] clone created successfully {{(pid=138473) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 27 11:15:49.632527 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.volume_utils [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Marking volume 570e326c-0f5d-44bd-a289-2e89c4df917f as bootable. {{(pid=138473) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 27 11:15:49.642722 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Copying metadata from image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 to 570e326c-0f5d-44bd-a289-2e89c4df917f. {{(pid=138473) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Sep 27 11:15:49.643190 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.flows.manager.create_volume [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Creating volume glance metadata for volume 570e326c-0f5d-44bd-a289-2e89c4df917f backed by image c1623d3a-c1bb-4649-9910-5deadb8a1ff6 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=138473) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Sep 27 11:15:49.698814 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (09b4d6bc-dd1b-4df6-aed5-d77776eff403) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-570e326c-0f5d-44bd-a289-2e89c4df917f', 'volume_size': 1, 'image_id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '7d515506957d41b7867232158c5ad868', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'created_at': datetime.datetime(2022, 9, 27, 9, 47, 12, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 27, 9, 47, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 11:15:49.703056 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a553abad-d6bf-4ef0-9c25-991e0a860367) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 27 11:15:49.713251 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.flows.manager.create_volume [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Volume volume-570e326c-0f5d-44bd-a289-2e89c4df917f (570e326c-0f5d-44bd-a289-2e89c4df917f): created successfully Sep 27 11:15:49.717002 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a553abad-d6bf-4ef0-9c25-991e0a860367) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=138473) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 27 11:15:49.722257 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Flow 'volume_create_manager' (9b793d8e-8cd7-4a66-97d9-fcb7b6160aa6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=138473) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 27 11:15:49.731702 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [None req-e61b2d24-3e4a-4fb4-886d-08b6cd78eadc tempest-TestShelveInstance-666939746 None] Created volume successfully. Sep 27 11:15:52.317066 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-5d532368-f821-43fc-a8b2-bfa911df8308 req-239aa8af-b14a-45e9-9eab-9113911109f4 tempest-TestShelveInstance-666939746 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 11:15:53.128671 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-5d532368-f821-43fc-a8b2-bfa911df8308 req-239aa8af-b14a-45e9-9eab-9113911109f4 tempest-TestShelveInstance-666939746 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.811s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 11:15:53.129581 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-5d532368-f821-43fc-a8b2-bfa911df8308 req-239aa8af-b14a-45e9-9eab-9113911109f4 tempest-TestShelveInstance-666939746 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-570e326c-0f5d-44bd-a289-2e89c4df917f', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '570e326c-0f5d-44bd-a289-2e89c4df917f', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 11:15:53.156473 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-5d532368-f821-43fc-a8b2-bfa911df8308 req-239aa8af-b14a-45e9-9eab-9113911109f4 tempest-TestShelveInstance-666939746 None] Connection info returned from driver {'name': 'volumes/volume-570e326c-0f5d-44bd-a289-2e89c4df917f', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '570e326c-0f5d-44bd-a289-2e89c4df917f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'ac2825d2-2948-40e2-8031-31778553c244'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 11:15:53.207290 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-5d532368-f821-43fc-a8b2-bfa911df8308 req-239aa8af-b14a-45e9-9eab-9113911109f4 tempest-TestShelveInstance-666939746 None] attachment_update completed successfully. Sep 27 11:16:07.619030 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=32,cinder:UPDATE=6,cinder:INSERT=5 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:16:27.622110 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=29,cinder:UPDATE=6,cinder:INSERT=7,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:16:30.054214 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:16:30.056241 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:16:30.088238 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:16:30.116964 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:16:47.623227 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:17:07.630044 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:17:27.630237 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:17:30.052921 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:17:30.053605 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:17:30.078129 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:17:30.103424 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:17:47.619147 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:18:07.619443 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:18:21.599325 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:18:21.610984 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 req-8c05df15-8e88-400e-8520-85a46e7cf663 tempest-TestShelveInstance-666939746 None] Terminate volume connection completed successfully. Sep 27 11:18:21.612031 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 req-8c05df15-8e88-400e-8520-85a46e7cf663 tempest-TestShelveInstance-666939746 None] Deleting attachment ac2825d2-2948-40e2-8031-31778553c244. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 11:18:23.375693 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 req-4ec8b604-2c50-4fae-91d6-cfb9d18bb174 tempest-TestShelveInstance-666939746 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 11:18:24.108093 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_concurrency.processutils [req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 req-4ec8b604-2c50-4fae-91d6-cfb9d18bb174 tempest-TestShelveInstance-666939746 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.732s {{(pid=138473) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 11:18:24.109001 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 req-4ec8b604-2c50-4fae-91d6-cfb9d18bb174 tempest-TestShelveInstance-666939746 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-570e326c-0f5d-44bd-a289-2e89c4df917f', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'a0a9ff1d-f350-4984-ac7b-a927bddf8534', 'volume_id': '570e326c-0f5d-44bd-a289-2e89c4df917f', 'discard': True}} {{(pid=138473) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Sep 27 11:18:24.137408 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 req-4ec8b604-2c50-4fae-91d6-cfb9d18bb174 tempest-TestShelveInstance-666939746 None] Connection info returned from driver {'name': 'volumes/volume-570e326c-0f5d-44bd-a289-2e89c4df917f', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '570e326c-0f5d-44bd-a289-2e89c4df917f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'd722311e-9b5d-43fb-aa43-4d2cf8875ccb'} {{(pid=138473) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 27 11:18:24.193095 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 req-4ec8b604-2c50-4fae-91d6-cfb9d18bb174 tempest-TestShelveInstance-666939746 None] attachment_update completed successfully. Sep 27 11:18:30.048640 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:18:30.049722 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:18:30.084762 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:18:30.129811 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:18:37.612970 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=13,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:18:47.617131 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=19,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:19:17.615053 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:19:17.619606 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:19:30.047232 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:19:30.048203 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:19:30.074007 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:19:30.103890 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:19:47.626083 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:19:57.612713 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:01.972642 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-9cb73c19-e5b1-4c0c-b7f7-24e1d2033076 tempest-TestShelveInstance-666939746 None] Terminate volume connection completed successfully. Sep 27 11:20:01.973356 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.manager [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-9cb73c19-e5b1-4c0c-b7f7-24e1d2033076 tempest-TestShelveInstance-666939746 None] Deleting attachment d722311e-9b5d-43fb-aa43-4d2cf8875ccb. {{(pid=138473) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 27 11:20:02.109944 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] Acquiring lock "cinder-570e326c-0f5d-44bd-a289-2e89c4df917f-delete_volume" by "delete_volume" {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 27 11:20:02.125367 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] Lock "cinder-570e326c-0f5d-44bd-a289-2e89c4df917f-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 27 11:20:02.169204 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:20:02.254457 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] volume has no backup snaps {{(pid=138473) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 27 11:20:02.263470 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] deleting rbd volume volume-570e326c-0f5d-44bd-a289-2e89c4df917f {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 27 11:20:02.486467 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] volume is a clone so cleaning references {{(pid=138473) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 27 11:20:02.518639 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] deleting parent snapshot volume-570e326c-0f5d-44bd-a289-2e89c4df917f.clone_snap {{(pid=138473) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 27 11:20:03.206390 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.quota [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] Created reservations ['9733bd4a-02c6-454d-80e0-b22c65adfa18', 'bcd08be8-81c6-43b6-981e-59b3f1f19822', '34bc17b5-659d-4baa-885e-c183919c3d9f', '1edafa35-8e1b-4fbd-88a1-f4aaf578ea6f'] {{(pid=138473) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 27 11:20:03.260876 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:20:03.318152 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:20:03.351507 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:20:03.356798 ncdv-master-1 cinder-volume[138473]: INFO cinder.volume.manager [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] Deleted volume successfully. Sep 27 11:20:03.372279 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.coordination [req-a329da7f-f36c-40a4-8f64-d850ae537a88 req-98eb90a9-b6da-45e0-a295-75fdba0398cf tempest-TestShelveInstance-666939746 None] Lock "cinder-570e326c-0f5d-44bd-a289-2e89c4df917f-delete_volume" released by "delete_volume" :: held 1.247s {{(pid=138473) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 27 11:20:17.618305 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=18,cinder:UPDATE=10,cinder:INSERT=2,cinder:DELETE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:17.623706 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=15,cinder:INSERT=2,cinder:UPDATE=8 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:30.050381 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:20:30.051420 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:20:30.086397 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:20:30.116930 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:20:37.629890 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:57.625893 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=10,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:21:17.622756 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=4,cinder:SELECT=6 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:21:30.053831 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:21:30.055159 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:21:30.079645 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:21:30.103430 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:21:37.628379 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:21:57.619643 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:17.626931 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:30.059194 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:22:30.060173 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:22:30.093637 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:22:30.117469 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:22:37.621649 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:37.622271 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:57.630302 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:23:17.624063 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:23:30.062766 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:23:30.063538 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:23:30.094672 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:23:30.128968 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:23:37.627880 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:23:47.634645 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:24:17.625207 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:24:27.624023 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:24:30.067627 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:24:30.068465 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:24:30.097303 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:24:30.130423 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:24:57.616457 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:07.618412 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:27.627557 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:27.628339 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:30.066090 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:25:30.066898 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:25:30.105429 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:25:30.148056 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:25:57.618554 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=1,cinder:SELECT=4 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:26:07.616481 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:26:30.067516 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:26:30.068712 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:26:30.101139 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:26:30.126231 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:26:37.619134 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:26:47.617767 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:27:17.619314 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:27:27.616105 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:27:30.072593 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:27:30.073737 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:27:30.110177 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:27:30.143692 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:27:57.610623 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:28:07.613581 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:28:30.070925 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:28:30.071972 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:28:30.104391 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:28:30.128737 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:28:37.610730 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:28:47.616690 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:29:07.623453 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:29:17.619593 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:29:30.074948 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:29:30.076009 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:29:30.111468 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:29:30.136533 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:29:47.619611 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=8,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:29:47.628314 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:30:17.627670 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:UPDATE=2,cinder:SELECT=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:30:27.624526 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:30:30.072532 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:30:30.073269 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:30:30.098836 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:30:30.126802 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:30:57.629717 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:31:07.624299 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:31:30.072753 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:31:30.073835 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:31:30.109292 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:31:30.133761 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:31:37.616963 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:31:47.617236 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:32:17.616103 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:32:27.616161 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:32:30.070049 ncdv-master-1 cinder-volume[138473]: DEBUG oslo_service.periodic_task [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=138473) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 27 11:32:30.070722 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:32:30.095020 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.volume.drivers.rbd [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=138473) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 27 11:32:30.118847 ncdv-master-1 cinder-volume[138473]: DEBUG cinder.manager [None req-b3231490-955d-4a31-b24c-48e8dabe0ad4 None None] Notifying Schedulers of capabilities ... {{(pid=138473) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 27 11:32:37.623784 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:33:07.614921 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:33:17.616139 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:33:47.615193 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:33:57.615103 ncdv-master-1 cinder-volume[138473]: DEBUG dbcounter [-] [138473] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=138473) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}