-- Logs begin at Sat 2022-10-01 20:01:15 UTC, end at Sat 2022-10-01 22:24:40 UTC. -- Oct 01 20:39:37.903072 ncdv-master-1 systemd[1]: Started Devstack devstack@c-vol.service. Oct 01 20:39:41.172509 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "singleton_lock" {{(pid=133222) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 01 20:39:41.173251 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=133222) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 01 20:39:41.173251 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=133222) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 01 20:39:41.598743 ncdv-master-1 cinder-volume[133222]: INFO dbcounter [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Registered counter for database cinder Oct 01 20:39:41.618033 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_db.sqlalchemy.engines [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133222) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 01 20:39:41.624603 ncdv-master-1 cinder-volume[133222]: DEBUG dbcounter [-] [133222] Writer thread running {{(pid=133222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 01 20:39:41.919722 ncdv-master-1 cinder-volume[133222]: INFO cinder.rpc [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version. Oct 01 20:39:41.924353 ncdv-master-1 cinder-volume[133222]: INFO cinder.rpc [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version. Oct 01 20:39:41.931784 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_db.sqlalchemy.engines [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133222) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 01 20:39:41.999902 ncdv-master-1 cinder-volume[133222]: INFO cinder.volume.manager [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Determined volume DB was empty at startup. Oct 01 20:39:42.000660 ncdv-master-1 cinder-volume[133222]: DEBUG cinder.volume.manager [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Cinder Volume DB check: vol_db_empty=True {{(pid=133222) __init__ /opt/stack/cinder/cinder/volume/manager.py:290}} Oct 01 20:39:42.009470 ncdv-master-1 cinder-volume[133222]: INFO cinder.volume.manager [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Service not found for updating active_backend_id, assuming default for driver init. Oct 01 20:39:42.010246 ncdv-master-1 cinder-volume[133222]: WARNING stevedore.named [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager Oct 01 20:39:42.010724 ncdv-master-1 cinder-volume[133222]: WARNING castellan.key_manager [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' Oct 01 20:39:42.011862 ncdv-master-1 cinder-volume[133222]: WARNING cinder.keymgr.conf_key_mgr [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] This key manager is insecure and is not recommended for production deployments: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' Oct 01 20:39:42.034578 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_db.sqlalchemy.engines [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133222) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 01 20:39:42.039617 ncdv-master-1 cinder-volume[133222]: INFO cinder.rpc [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version. Oct 01 20:39:42.043977 ncdv-master-1 cinder-volume[133222]: INFO cinder.rpc [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Automatically selected cinder-volume RPC version 3.18 as minimum service version. Oct 01 20:39:42.044665 ncdv-master-1 cinder-volume[133222]: WARNING stevedore.named [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager Oct 01 20:39:42.045355 ncdv-master-1 cinder-volume[133222]: WARNING castellan.key_manager [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' Oct 01 20:39:42.046803 ncdv-master-1 cinder-volume[133222]: INFO cinder.volume.manager [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Image-volume cache enabled for host ncdv-master-1@ceph. Oct 01 20:39:42.052671 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_db.sqlalchemy.engines [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133222) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 01 20:39:42.076903 ncdv-master-1 cinder-volume[133222]: INFO oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Starting 1 workers Oct 01 20:39:42.082151 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Started child 133391 {{(pid=133222) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}} Oct 01 20:39:42.083969 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Full set of CONF: {{(pid=133222) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}} Oct 01 20:39:42.084515 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ******************************************************************************** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 01 20:39:42.084830 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] Configuration options gathered from: {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 01 20:39:42.085421 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 01 20:39:42.086213 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 01 20:39:42.086708 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ================================================================================ {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 01 20:39:42.087584 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] allow_availability_zone_fallback = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.088117 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] allow_compression_on_image_upload = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.088782 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] allowed_direct_url_schemes = [] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.089322 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] api_rate_limit = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.089964 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] auth_strategy = keystone {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.090582 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] az_cache_duration = 3600 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.090763 ncdv-master-1 cinder-volume[133391]: INFO cinder.service [-] Starting cinder-volume node (version 21.1.0) Oct 01 20:39:42.091687 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_availability_zone = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.092231 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_host = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.092901 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_name = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.093473 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_stats_polling_interval = 60 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.094139 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backup_api_class = cinder.backup.api.API {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.094826 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backup_manager = cinder.backup.manager.BackupManager {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.095406 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backup_name_template = backup-%s {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.095961 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backup_use_same_host = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.096482 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backup_use_temp_snapshot = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.097035 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] chap_password = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.097573 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] chap_username = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.098103 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.098706 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cinder_internal_tenant_project_id = b7875946723844198353bbfcf22516fb {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.099246 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cinder_internal_tenant_user_id = 197a0ae6b71f4635a069ac0238e017c9 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.099782 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cloned_volume_same_az = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.100313 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cluster = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.100964 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] compression_format = gzip {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.101526 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] compute_api_class = cinder.compute.nova.API {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.102140 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] config_dir = [] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.102623 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] config_file = ['/etc/cinder/cinder.conf'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.103159 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] config_source = [] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.103697 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] consistencygroup_api_class = cinder.consistencygroup.api.API {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.104413 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] control_exchange = openstack {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.105085 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] debug = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.105590 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] default_availability_zone = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.106100 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] default_group_type = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.106972 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.107533 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] default_volume_type = ceph {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.108057 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] driver_client_cert = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.108567 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] driver_client_cert_key = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.109073 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] driver_data_namespace = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.109570 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] driver_ssl_cert_path = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.110220 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] driver_ssl_cert_verify = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.110776 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] driver_use_ssl = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.111308 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] enable_force_upload = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.111792 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] enable_new_services = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.112307 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] enable_unsupported_driver = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.113449 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] enabled_backends = ['ceph'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.113991 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] enforce_multipath_for_image_xfer = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.114631 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] filter_function = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.115268 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] glance_api_insecure = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.115802 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] glance_api_servers = ['http://10.222.0.17/image'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.116307 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] glance_api_ssl_compression = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.116822 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] glance_ca_certificates_file = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.117366 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] glance_catalog_info = image:glance:publicURL {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.117887 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] glance_certfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.118699 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] glance_core_properties = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.118801 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] glance_keyfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.119144 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] glance_num_retries = 3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.119680 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] glance_request_timeout = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.120180 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] goodness_function = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.120715 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] graceful_shutdown_timeout = 5 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.121228 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] group_api_class = cinder.group.api.API {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.121677 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] host = ncdv-master-1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.122194 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] iet_conf = /etc/iet/ietd.conf {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.122780 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] image_compress_on_upload = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.123314 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] image_conversion_address_space_limit = 1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.123843 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] image_conversion_cpu_limit = 60 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.125288 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] image_conversion_dir = /opt/stack/data/cinder/conversion {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.125810 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] image_conversion_disable = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.126341 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] image_upload_use_cinder_backend = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.126877 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] image_upload_use_internal_tenant = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.128036 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] image_volume_cache_enabled = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.128617 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] image_volume_cache_max_count = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.129730 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] image_volume_cache_max_size_gb = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.130344 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] init_host_max_objects_retrieval = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.130963 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] instance_format = [instance: %(uuid)s] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.131542 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.132523 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] iscsi_iotype = fileio {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.132860 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] iscsi_secondary_ip_addresses = [] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.133208 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Starting volume driver RBDDriver (1.3.0) Oct 01 20:39:42.133673 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] iscsi_target_flags = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.134139 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] iscsi_write_cache = on {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.135193 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] iser_helper = tgtadm {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.136015 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] iser_ip_address = 10.222.0.17 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.136561 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] iser_port = 3260 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.136819 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:39:42.137131 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.137626 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] log_config_append = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.138116 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.138610 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] log_dir = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.139104 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] log_file = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.139507 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] log_options = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.139974 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] log_rotate_interval = 1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.141003 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] log_rotate_interval_type = days {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.141588 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] log_rotation_type = none {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.142281 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.142827 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.143429 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.144013 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.144524 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.145922 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] max_age = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.146521 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] max_logfile_count = 30 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.147092 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] max_logfile_size_mb = 200 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.147707 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] max_over_subscription_ratio = 20.0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.152348 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] message_reap_interval = 86400 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.153012 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] message_ttl = 2592000 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.153653 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] migration_create_volume_timeout_secs = 300 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.154249 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] monkey_patch = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.154875 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] monkey_patch_modules = [] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.155410 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] my_ip = 10.222.0.17 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.156181 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] no_snapshot_gb_quota = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.156773 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] num_iser_scan_tries = 3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.157392 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] num_shell_tries = 3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.158011 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] num_volume_device_scan_tries = 3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.158676 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] nvmet_ns_id = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.159251 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] nvmet_port_id = 1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.159865 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] osapi_max_limit = 1000 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.160239 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] osapi_volume_ext_list = [] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.160726 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] osapi_volume_extension = ['cinder.api.contrib.standard_extensions'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.161100 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] osapi_volume_listen = 0.0.0.0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.161480 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] osapi_volume_listen_port = 8776 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.161851 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] osapi_volume_use_ssl = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.162219 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] osapi_volume_workers = 2 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.162882 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] per_volume_size_limit = -1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.163242 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] periodic_fuzzy_delay = 60 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.163612 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] periodic_interval = 60 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.164027 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] project_id_regex = [0-9a-f\-]+ {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.164375 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] public_endpoint = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.164725 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] publish_errors = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.165113 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] quota_backup_gigabytes = 1000 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.165506 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] quota_backups = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.165901 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] quota_consistencygroups = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.166284 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] quota_driver = cinder.quota.DbQuotaDriver {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.166665 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] quota_gigabytes = 1000 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.167034 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] quota_groups = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.167654 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] quota_snapshots = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.168032 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] quota_volumes = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.168546 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] rate_limit_burst = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.169146 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] rate_limit_except_level = CRITICAL {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.169748 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] rate_limit_interval = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.170307 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] reinit_driver_count = 3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.170833 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] replication_device = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.171403 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] report_discard_supported = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.171917 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] report_interval = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.172719 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] reservation_clean_interval = 86400 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.173684 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] reservation_expire = 86400 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.174237 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] reserved_percentage = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.174861 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] resource_query_filters_file = /etc/cinder/resource_filters.json {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.175432 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] rootwrap_config = /etc/cinder/rootwrap.conf {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.176051 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] rpc_response_timeout = 120 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.176644 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] run_external_periodic_tasks = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.177995 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] scheduler_manager = cinder.scheduler.manager.SchedulerManager {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.178589 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] scst_target_driver = iscsi {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.179118 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] scst_target_iqn_name = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.179422 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] service_down_time = 60 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.179796 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] snapshot_name_template = snapshot-%s {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.180169 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] snapshot_same_host = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.180524 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] split_loggers = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.180820 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] state_path = /opt/stack/data/cinder {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.181211 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] storage_availability_zone = nova {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.181561 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] storage_protocol = iSCSI {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.181941 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] syslog_log_facility = LOG_USER {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.182305 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] target_helper = lioadm {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.182914 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] target_ip_address = 10.222.0.17 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.183274 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] target_port = 3260 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.183638 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] target_prefix = iqn.2010-10.org.openstack: {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.184183 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] target_protocol = iscsi {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.184690 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] trace_flags = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.185540 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] transfer_api_class = cinder.transfer.api.API {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.186122 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] transport_url = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.186698 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] until_refresh = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.187231 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] use_chap_auth = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.187763 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] use_default_quota_class = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.188334 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] use_eventlog = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.188488 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:39:42.189023 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] use_journal = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.189558 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] use_json = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.190107 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] use_multipath_for_image_xfer = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.190698 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] use_stderr = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.191244 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] use_syslog = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.192333 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] verify_glance_signatures = enabled {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.193127 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_api_class = cinder.volume.api.API {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.193654 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_backend_name = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.194216 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_clear = zero {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.194766 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_clear_ionice = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.195311 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_clear_size = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.196125 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.196662 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_copy_bps_limit = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.197201 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_dd_blocksize = 1M {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.197758 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_manager = cinder.volume.manager.VolumeManager {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.198330 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_name_template = volume-%s {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.198915 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_service_inithost_offload = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.199493 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volume_usage_audit_period = month {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.200209 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] volumes_dir = /opt/stack/data/cinder/volumes {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.200741 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] watch_log_file = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.201307 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] zoning_mode = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 01 20:39:42.201808 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] os_brick.lock_path = /opt/stack/data/cinder {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.202249 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.202680 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.203185 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_concurrency.disable_process_locking = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.203582 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_concurrency.lock_path = /opt/stack/data/cinder {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.203988 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.204351 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.204715 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.205097 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.205450 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.205908 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.admin_password = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.206276 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.206651 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.admin_token = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.207173 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.admin_user = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.208337 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.auth_admin_prefix = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.208880 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.209427 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.auth_port = 35357 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.209997 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.auth_protocol = https {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.210668 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.auth_section = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.211216 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.auth_type = password {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.211600 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.auth_uri = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.211954 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.auth_version = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.212335 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.cache = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.212673 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.213053 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.certfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.213411 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.delay_auth_decision = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.213825 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.214173 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.http_connect_timeout = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.214554 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.215258 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.identity_uri = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.215784 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.include_service_catalog = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.216303 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.insecure = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.216844 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.interface = public {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.217198 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.keyfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.217581 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.217937 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.218290 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.218674 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.219033 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.219379 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.219765 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.220123 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.220770 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.221125 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.region_name = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.221528 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.221889 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.service_token_roles_required = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.222252 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.service_type = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.222632 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.token_cache_time = 300 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.223278 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.223867 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.backend = dogpile.cache.null {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.224407 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.backend_argument = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.224932 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.config_prefix = cache.oslo {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.225374 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.dead_timeout = 60.0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.225763 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.debug_cache_backend = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.226118 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.enable_retry_client = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.226479 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.enable_socket_keepalive = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.226847 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.enabled = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.227200 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.expiration_time = 600 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.227555 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.hashclient_retry_attempts = 2 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.227921 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.hashclient_retry_delay = 1.0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.228294 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.memcache_dead_retry = 300 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.228663 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.memcache_password = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.229051 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.229414 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.229749 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.memcache_pool_maxsize = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.230106 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.230478 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.memcache_sasl_enabled = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.232669 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.memcache_servers = ['localhost:11211'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.233398 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.memcache_socket_timeout = 1.0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.234210 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.memcache_username = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.235149 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.proxies = [] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.235525 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.retry_attempts = 2 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.235900 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.retry_delay = 0.0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.236263 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.socket_keepalive_count = 1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.236630 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.socket_keepalive_idle = 1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.236987 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.socket_keepalive_interval = 1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.237338 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.tls_allowed_ciphers = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.237687 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.tls_cafile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.238034 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.tls_certfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.238392 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.tls_enabled = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.238797 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cache.tls_keyfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.239189 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] coordination.backend_url = etcd3+http://10.222.0.17:2379 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.239632 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.backend = sqlalchemy {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.239948 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.connection = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.240350 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.connection_debug = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.240720 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.connection_parameters = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.241094 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.connection_recycle_time = 3600 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.241471 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.connection_trace = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.241853 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.db_inc_retry_interval = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.242212 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.db_max_retries = 20 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.242661 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.db_max_retry_interval = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.243554 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.db_retry_interval = 1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.244073 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.max_overflow = 50 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.244665 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.max_pool_size = 5 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.245209 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.max_retries = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.245716 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.mysql_enable_ndb = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.246253 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.246809 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.pool_timeout = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.247318 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.retry_interval = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.247841 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.slave_connection = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.248203 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.sqlite_synchronous = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.248571 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] database.use_db_reconnect = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.248975 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_policy.enforce_new_defaults = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.249350 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_policy.enforce_scope = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.249741 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_policy.policy_default_rule = default {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.250140 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.250619 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_policy.policy_file = policy.yaml {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.251039 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.251383 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.251928 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.252294 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.252666 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.253113 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] profiler.connection_string = messaging:// {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.253414 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] profiler.enabled = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.253778 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] profiler.es_doc_type = notification {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.254668 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] profiler.es_scroll_size = 10000 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.255038 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] profiler.es_scroll_time = 2m {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.255406 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] profiler.filter_error_trace = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.255822 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] profiler.hmac_keys = SECRET_KEY {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.256184 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] profiler.sentinel_service_name = mymaster {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.256579 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] profiler.socket_timeout = 0.1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.256931 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] profiler.trace_sqlalchemy = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.257345 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] privsep_osbrick.capabilities = [21] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.257888 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] privsep_osbrick.group = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.258257 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] privsep_osbrick.helper_command = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.258661 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] privsep_osbrick.logger_name = oslo_privsep.daemon {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.259026 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.259395 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] privsep_osbrick.user = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.259788 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cinder_sys_admin.capabilities = [0, 1, 2, 3, 12, 21] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.260136 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cinder_sys_admin.group = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.260478 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cinder_sys_admin.helper_command = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.260829 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cinder_sys_admin.logger_name = oslo_privsep.daemon {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.261180 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.261561 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] cinder_sys_admin.user = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.261927 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] service_user.auth_section = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.262275 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] service_user.auth_type = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.262665 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] service_user.cafile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.263038 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] service_user.certfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.263397 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] service_user.collect_timing = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.263753 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] service_user.insecure = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.264085 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] service_user.keyfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.264600 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] service_user.send_service_user_token = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.264984 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] service_user.split_loggers = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.265338 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] service_user.timeout = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.265767 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.backend_availability_zone = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.266116 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.backend_native_threads_pool_size = 20 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.266463 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.backup_use_temp_snapshot = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.266865 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.chap_password = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.267231 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.chap_username = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.267614 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.267988 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.268342 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.268682 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.driver_client_cert = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.269026 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.driver_client_cert_key = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.269366 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.driver_data_namespace = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.269754 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.270705 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.271052 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.driver_use_ssl = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.271402 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.enable_deferred_deletion = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.271749 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.enable_unsupported_driver = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.272098 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.272454 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.extra_capabilities = {} {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.272800 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.filter_function = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.273140 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.goodness_function = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.273495 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.iet_conf = /etc/iet/ietd.conf {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.273844 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.274195 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.274628 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.274976 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.275326 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.275692 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.iscsi_iotype = fileio {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.276046 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.276401 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.iscsi_target_flags = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.276785 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.iscsi_write_cache = on {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.277199 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.iser_helper = tgtadm {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.277735 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.iser_ip_address = 10.222.0.17 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.278087 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.iser_port = 3260 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.278443 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.278852 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.279207 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.279555 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.num_shell_tries = 3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.279907 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.280255 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.nvmet_ns_id = 10 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.280601 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.nvmet_port_id = 1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.280957 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.281327 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rados_connection_interval = 5 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.281655 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rados_connection_retries = 3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.282014 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rbd_ceph_conf = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.282377 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.282749 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.283507 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.283507 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.283850 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rbd_pool = rbd {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.284141 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rbd_secret_uuid = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.284513 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.284847 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.rbd_user = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.285290 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.285662 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.replication_device = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.286014 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.report_discard_supported = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.286363 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.286744 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.reserved_percentage = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.287109 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.scst_target_driver = iscsi {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.287451 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.scst_target_iqn_name = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.287815 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.storage_protocol = iSCSI {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.288171 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.suppress_requests_ssl_warnings = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.288531 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.target_helper = tgtadm {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.289008 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.target_ip_address = 10.222.0.17 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.289383 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.target_port = 3260 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.289750 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.290118 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.target_protocol = iscsi {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.290460 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.trace_flags = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.290835 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.unique_fqdn_network = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.291182 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.use_chap_auth = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.291529 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.291883 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.volume_backend_name = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.292233 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.volume_clear = zero {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.292575 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.volume_clear_ionice = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.293028 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.volume_clear_size = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.293396 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.293831 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.295054 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.295330 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:39:42.295435 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.295909 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] backend_defaults.volumes_dir = /opt/stack/data/cinder/volumes {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.296308 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_reports.file_event_handler = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.296727 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.297111 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_reports.log_dir = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.297500 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.backend_availability_zone = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.297857 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.backend_host = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.298205 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.backend_native_threads_pool_size = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.298573 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.backup_use_temp_snapshot = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.298924 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.chap_password = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.299270 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.chap_username = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.299618 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.chiscsi_conf = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.299984 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.deferred_deletion_delay = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.300331 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.deferred_deletion_purge_interval = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.300681 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.driver_client_cert = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.301045 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.driver_client_cert_key = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.301393 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.driver_data_namespace = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.301747 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.driver_ssl_cert_path = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.302091 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.driver_ssl_cert_verify = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.302858 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.driver_use_ssl = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.302858 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.enable_deferred_deletion = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.303157 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.enable_unsupported_driver = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.303484 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.enforce_multipath_for_image_xfer = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.303845 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.extra_capabilities = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.304191 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.filter_function = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.304533 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.goodness_function = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.304883 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.iet_conf = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.305227 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.image_upload_use_cinder_backend = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.305594 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.image_upload_use_internal_tenant = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.305954 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.image_volume_cache_enabled = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.306606 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.image_volume_cache_max_count = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.306709 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.image_volume_cache_max_size_gb = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.306987 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.iscsi_iotype = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.307934 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.iscsi_secondary_ip_addresses = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.307934 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.iscsi_target_flags = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.308120 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.iscsi_write_cache = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.308378 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.iser_helper = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.308682 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.iser_ip_address = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.309035 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.iser_port = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.309375 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.iser_target_prefix = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.309719 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.max_over_subscription_ratio = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.310061 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.num_iser_scan_tries = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.310409 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.num_shell_tries = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.310778 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.num_volume_device_scan_tries = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.311121 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.nvmet_ns_id = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.311461 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.nvmet_port_id = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.311826 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rados_connect_timeout = -1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.312169 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rados_connection_interval = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.312509 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rados_connection_retries = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.312867 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rbd_ceph_conf = /etc/ceph/ceph.conf {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.313231 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rbd_cluster_name = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.313576 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rbd_exclusive_cinder_pool = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.314527 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rbd_flatten_volume_from_snapshot = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.314527 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rbd_max_clone_depth = 5 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.314717 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rbd_pool = volumes {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.315049 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rbd_secret_uuid = b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.315609 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rbd_store_chunk_size = 4 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.316186 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.rbd_user = cinder {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.317203 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.replication_connect_timeout = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.317588 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.replication_device = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.318157 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.report_discard_supported = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.318283 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.report_dynamic_total_capacity = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.318638 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.reserved_percentage = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.319059 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.scst_target_driver = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.319318 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.scst_target_iqn_name = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.320463 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.storage_protocol = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.320463 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.suppress_requests_ssl_warnings = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.320463 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.target_helper = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.320988 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.target_ip_address = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.321256 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.target_port = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.322395 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.target_prefix = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.322395 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.target_protocol = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.322395 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.trace_flags = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.322711 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.unique_fqdn_network = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.324005 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.use_chap_auth = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.324005 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.use_multipath_for_image_xfer = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.324005 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.volume_backend_name = ceph {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.324005 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.volume_clear = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.324320 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.volume_clear_ionice = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.324972 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.volume_clear_size = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.324972 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.volume_copy_blkio_cgroup_name = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.326318 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.volume_copy_bps_limit = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.326318 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.volume_dd_blocksize = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.326318 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.volume_driver = cinder.volume.drivers.rbd.RBDDriver {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.326318 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ceph.volumes_dir = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.327442 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.327442 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.327582 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.327863 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.328255 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.328604 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.329041 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.329392 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.329741 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.330553 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.330553 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.330859 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.331190 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.331535 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.331878 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.332266 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.333151 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.333496 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.333838 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.334207 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.334575 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.334930 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.335477 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.335838 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.336482 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.336859 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.ssl = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.337544 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.337831 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.338167 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.338551 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.339082 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_rabbit.ssl_version = {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.339680 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_notifications.driver = [] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.340050 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_notifications.retry = -1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.340481 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.340853 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] oslo_messaging_notifications.transport_url = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.341202 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] key_manager.backend = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.341496 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] key_manager.fixed_key = **** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.341934 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.342284 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.barbican_api_version = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.342679 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.barbican_endpoint = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.343075 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.barbican_endpoint_type = public {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.343642 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.barbican_region_name = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.344183 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.cafile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.344535 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.certfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.344917 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.collect_timing = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.345364 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.insecure = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.345723 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.keyfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.346086 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.number_of_retries = 60 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.346436 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.retry_delay = 1 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.346828 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.send_service_user_token = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.347181 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.split_loggers = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.347294 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.driver [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Driver hasn't implemented _init_vendor_properties() Oct 01 20:39:42.347525 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.timeout = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.347626 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.driver [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Initialized capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.3.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:3f02706e-6a89-4cc7-a5d0-346f6e188c25:cinder:volumes', 'backend_state': 'up', 'qos_support': True, 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=133391) init_capabilities /opt/stack/cinder/cinder/volume/driver.py:812}} Oct 01 20:39:42.347896 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.verify_ssl = True {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.348239 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican.verify_ssl_path = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.348659 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican_service_user.auth_section = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.349000 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican_service_user.auth_type = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.349342 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican_service_user.cafile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.349711 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican_service_user.certfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.350056 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican_service_user.collect_timing = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.350400 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican_service_user.insecure = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.350764 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican_service_user.keyfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.351115 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican_service_user.split_loggers = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.351472 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] barbican_service_user.timeout = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.351830 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.approle_role_id = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.352171 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.approle_secret_id = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.352512 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.cafile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.352850 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.certfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.353198 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.collect_timing = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.353542 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.insecure = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.353888 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.keyfile = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.354311 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.kv_mountpoint = secret {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.354700 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.kv_version = 2 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.355045 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.namespace = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.355387 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.root_token_id = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.355731 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.split_loggers = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.356069 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.ssl_ca_crt_file = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.356410 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.timeout = None {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.356757 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.use_ssl = False {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.357126 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 01 20:39:42.357407 ncdv-master-1 cinder-volume[133222]: DEBUG oslo_service.service [None req-a75a7f97-b217-4c43-bdc3-6642fb59f805 None None] ******************************************************************************** {{(pid=133222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 01 20:39:42.364381 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_db.sqlalchemy.engines [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133391) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 01 20:39:42.366789 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writer thread running {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 01 20:39:42.402088 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.db.sqlalchemy.api [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Building query based on filter {{(pid=133391) _process_snaps_filters /opt/stack/cinder/cinder/db/sqlalchemy/api.py:3885}} Oct 01 20:39:42.422153 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.image_utils [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Configuration image_conversion_dir is None or the path doesn't exist. {{(pid=133391) cleanup_temporary_file /opt/stack/cinder/cinder/image/image_utils.py:991}} Oct 01 20:39:42.428825 ncdv-master-1 cinder-volume[133391]: INFO cinder.keymgr.migration [None req-f0d40501-3d67-4ef1-9e31-0dcdb7f50166 None None] Not migrating encryption keys because the ConfKeyManager is still in use. Oct 01 20:39:42.436178 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:39:42.469775 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:39:42.498003 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:39:42.522263 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Driver initialization completed successfully. Oct 01 20:39:42.523058 ncdv-master-1 cinder-volume[133391]: INFO cinder.manager [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Initiating service 3 cleanup Oct 01 20:39:42.532997 ncdv-master-1 cinder-volume[133391]: INFO cinder.manager [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Service 3 cleanup completed. Oct 01 20:39:42.533378 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.service [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Creating RPC server for service cinder-volume {{(pid=133391) start /opt/stack/cinder/cinder/service.py:238}} Oct 01 20:39:42.543903 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.service [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=133391) start /opt/stack/cinder/cinder/service.py:244}} Oct 01 20:39:42.592764 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Initializing RPC dependent components of volume driver RBDDriver (1.3.0) Oct 01 20:39:42.594378 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:39:42.634896 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:39:42.681495 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-763657e2-26ca-4716-8be2-b1a840fcefc4 None None] Driver post RPC initialization completed successfully. Oct 01 20:39:52.064647 ncdv-master-1 cinder-volume[133222]: DEBUG dbcounter [-] [133222] Writing DB stats cinder:SELECT=16,cinder:INSERT=1 {{(pid=133222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:52.674141 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=22,cinder:INSERT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:39:52.676779 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:32.683962 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=10,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:32.685510 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:40:52.683394 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:12.691239 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:22.697059 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:23.684250 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:41:23.685009 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:41:23.709710 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:41:23.733104 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:41:32.703110 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:41:52.711822 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:42:02.713547 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:42:12.721825 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:42:23.681755 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:42:23.682417 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:42:23.707024 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:42:23.729148 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:42:32.721905 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:42:32.722732 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:42:52.728974 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=1,cinder:SELECT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:02.732926 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:22.735102 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:23.684664 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:43:23.685407 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:43:23.713002 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:43:23.740721 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:43:32.740172 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:43:42.748559 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:44:02.750865 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:44:02.751607 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:44:12.761552 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:44:23.687752 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:44:23.688403 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:44:23.711187 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:44:23.737478 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:44:32.763476 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:44:32.764108 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:44:52.760201 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:45:12.766665 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:45:22.770021 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:45:23.693006 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:45:23.693951 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:45:23.717673 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:45:23.739977 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:45:42.776715 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:45:52.782916 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:46:12.792735 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:46:12.793428 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:46:23.694688 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:46:23.695372 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:46:23.718033 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:46:23.739150 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:46:32.794299 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=1,cinder:SELECT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:46:42.802960 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:46:52.805960 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:47:02.808804 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:47:22.807791 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:47:23.693829 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:47:23.694531 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:47:23.719717 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:47:23.742511 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:47:32.810817 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:47:52.815885 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:02.823016 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:22.822857 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:23.698709 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:48:23.699489 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:48:23.727260 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:48:23.763216 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:48:42.837437 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:48:52.833615 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:22.821743 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:49:23.700589 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:49:23.701427 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:49:23.740229 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:49:23.807351 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:49:42.829800 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:02.844226 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:02.844995 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:22.842538 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:50:23.699033 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:50:23.699837 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:50:23.801839 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:50:23.878213 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:50:42.831593 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:02.831293 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:04.927435 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 20:51:04.988439 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Flow 'volume_create_manager' (35e8156e-5ca0-4b23-86dd-11ee83ea8090) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:51:04.996470 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (370aca7c-4caa-4371-973e-08e358f76f4f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:05.061800 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (370aca7c-4caa-4371-973e-08e358f76f4f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T20:51:04Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-177886056',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ea35f176-da9d-49b3-9ff6-4dce9e05b3c8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='bb36a4c325b54c8a964113ba5e52f47b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T20:51:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T20:51:05Z,use_quota=True,user_id='2aa4f468a15546eb9f72cb2b632e67bb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:05.068021 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eb047d71-8568-4c44-a2f4-975bfafba3d5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:05.072358 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eb047d71-8568-4c44-a2f4-975bfafba3d5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:05.077580 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b3acb746-da77-4aba-a614-31a5d2aa0952) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:05.240480 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b3acb746-da77-4aba-a614-31a5d2aa0952) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'ea35f176-da9d-49b3-9ff6-4dce9e05b3c8', 'volume_name': 'volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:05.259192 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (83b25304-eaa4-4805-bddd-1c4673287f48) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:05.279993 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (83b25304-eaa4-4805-bddd-1c4673287f48) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:05.299207 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4dc395b6-eb3c-435d-aa67-74e4553bcc94) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:05.304806 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume ea35f176-da9d-49b3-9ff6-4dce9e05b3c8: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 20:51:05.306300 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Cloning ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 20:51:05.308675 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 20:51:05.328251 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.019s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 20:51:05.354257 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Preparing cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:891}} Oct 01 20:51:05.357670 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 20:51:12.861909 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=9,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:23.703511 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:51:23.704218 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:51:23.809230 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:51:23.905404 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:51:25.261321 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.image_utils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Image fetch details: dest /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph, size 616.00 MB, duration 19.89 sec {{(pid=133391) fetch /opt/stack/cinder/cinder/image/image_utils.py:587}} Oct 01 20:51:25.262704 ncdv-master-1 cinder-volume[133391]: INFO cinder.image.image_utils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Image download 616.00 MB at 30.98 MB/s Oct 01 20:51:25.344520 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:51:25.589598 ncdv-master-1 sudo[144936]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph Oct 01 20:51:25.591979 ncdv-master-1 sudo[144936]: pam_unix(sudo:session): session opened for user root by (uid=0) Oct 01 20:51:28.268807 ncdv-master-1 sudo[144936]: pam_unix(sudo:session): session closed for user root Oct 01 20:51:28.271735 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph" returned: 0 in 2.926s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:51:28.273495 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.image_utils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Temporary image 90f3f35e-76ec-4c31-b581-329b315fe607 is fetched for user 2aa4f468a15546eb9f72cb2b632e67bb. {{(pid=133391) fetch /opt/stack/cinder/cinder/image/image_utils.py:1124}} Oct 01 20:51:28.429145 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:51:28.728425 ncdv-master-1 sudo[144958]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph Oct 01 20:51:28.730105 ncdv-master-1 sudo[144958]: pam_unix(sudo:session): session opened for user root by (uid=0) Oct 01 20:51:31.623734 ncdv-master-1 sudo[144958]: pam_unix(sudo:session): session closed for user root Oct 01 20:51:31.626032 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph" returned: 0 in 3.199s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:51:31.627078 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] creating volume 'volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 20:51:31.628716 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:51:31.873598 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 20:51:31.874778 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 20:51:31.913284 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Attempting download of 90f3f35e-76ec-4c31-b581-329b315fe607 ((None, None)) to volume ea35f176-da9d-49b3-9ff6-4dce9e05b3c8. {{(pid=133391) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1190}} Oct 01 20:51:32.055025 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_download_90f3f35e-76ec-4c31-b581-329b315fe607_anhorg3j {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:51:32.284118 ncdv-master-1 sudo[145117]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_download_90f3f35e-76ec-4c31-b581-329b315fe607_anhorg3j Oct 01 20:51:32.290325 ncdv-master-1 sudo[145117]: pam_unix(sudo:session): session opened for user root by (uid=0) Oct 01 20:51:35.261676 ncdv-master-1 sudo[145117]: pam_unix(sudo:session): session closed for user root Oct 01 20:51:35.263684 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_download_90f3f35e-76ec-4c31-b581-329b315fe607_anhorg3j" returned: 0 in 3.213s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:51:35.264836 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:51:35.560698 ncdv-master-1 sudo[145141]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph Oct 01 20:51:35.568148 ncdv-master-1 sudo[145141]: pam_unix(sudo:session): session opened for user root by (uid=0) Oct 01 20:51:38.210336 ncdv-master-1 sudo[145141]: pam_unix(sudo:session): session closed for user root Oct 01 20:51:38.212504 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph" returned: 0 in 2.947s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:51:38.214065 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.image_utils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] 90f3f35e-76ec-4c31-b581-329b315fe607 was vhdx, converting to raw {{(pid=133391) fetch_to_volume_format /opt/stack/cinder/cinder/image/image_utils.py:825}} Oct 01 20:51:38.215470 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmp1eefreum {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:51:38.296771 ncdv-master-1 sudo[145230]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmp1eefreum Oct 01 20:51:38.299943 ncdv-master-1 sudo[145230]: pam_unix(sudo:session): session opened for user root by (uid=0) Oct 01 20:51:41.779822 ncdv-master-1 sudo[145230]: pam_unix(sudo:session): session closed for user root Oct 01 20:51:41.782384 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmp1eefreum" returned: 0 in 3.566s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:51:41.784264 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:51:41.934778 ncdv-master-1 sudo[145286]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph Oct 01 20:51:41.938284 ncdv-master-1 sudo[145286]: pam_unix(sudo:session): session opened for user root by (uid=0) Oct 01 20:51:42.835664 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=15,cinder:UPDATE=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:43.882058 ncdv-master-1 sudo[145286]: pam_unix(sudo:session): session closed for user root Oct 01 20:51:43.884072 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info --output=json /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph" returned: 0 in 2.100s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:51:43.885943 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.image_utils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Image conversion details: src /opt/stack/data/cinder/conversion/image_fetch_90f3f35e-76ec-4c31-b581-329b315fe607_dk2a3u_8ncdv-master-1@ceph, size 700.00 MB, duration 3.57 sec, destination /opt/stack/data/cinder/conversion/tmp1eefreum {{(pid=133391) _convert_image /opt/stack/cinder/cinder/image/image_utils.py:419}} Oct 01 20:51:43.886734 ncdv-master-1 cinder-volume[133391]: INFO cinder.image.image_utils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Converted 700.00 MB image at 196.15 MB/s Oct 01 20:51:43.898707 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:51:44.088997 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 20:51:44.096950 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 20:51:44.121016 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] deleting rbd volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 20:51:44.532196 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Running cmd (subprocess): rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmp1eefreum volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:51:50.168645 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] CMD "rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmp1eefreum volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 5.636s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:51:50.289338 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:51:50.471644 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 20:51:50.472036 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 20:51:50.472495 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Downloaded image 90f3f35e-76ec-4c31-b581-329b315fe607 ((None, None)) to volume ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 successfully. {{(pid=133391) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1232}} Oct 01 20:51:50.531889 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.image_utils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Temporary image 90f3f35e-76ec-4c31-b581-329b315fe607 for user 2aa4f468a15546eb9f72cb2b632e67bb is deleted. {{(pid=133391) fetch /opt/stack/cinder/cinder/image/image_utils.py:1128}} Oct 01 20:51:50.538526 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:INSERT=1,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:51:50.650030 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Deprecated: Default quota for resource: volumes_ceph is set by the default quota flag: quota_volumes_ceph, it is now deprecated. Please use the default quota class for default quota. Oct 01 20:51:50.650898 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Deprecated: Default quota for resource: gigabytes_ceph is set by the default quota flag: quota_gigabytes_ceph, it is now deprecated. Please use the default quota class for default quota. Oct 01 20:51:50.830982 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Created reservations ['a76f8f1b-1bee-4344-96f5-d93f7651f234', '4b920bdf-4b20-4cab-a494-6b29282af2e8', '529fce6f-3077-4e6b-abb2-05d219221302', '98058745-195c-47d8-8c3a-2e31d7e0be47'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 20:51:50.856576 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Creating image volume entry: {'replication_extended_status': None, 'previous_status': None, 'group_id': None, 'terminated_at': None, 'deleted': False, 'metadata': {}, 'cluster_name': None, 'bootable': False, 'replication_driver_data': None, 'deleted_at': None, 'provider_geometry': None, 'updated_at': datetime.datetime(2022, 10, 1, 20, 51, 5, tzinfo=datetime.timezone.utc), 'volume_admin_metadata': [], 'multiattach': False, 'ec2_id': None, 'encryption_key_id': None, 'provider_auth': None, 'availability_zone': 'nova', 'created_at': datetime.datetime(2022, 10, 1, 20, 51, 4, tzinfo=datetime.timezone.utc), 'scheduled_at': datetime.datetime(2022, 10, 1, 20, 51, 5, tzinfo=datetime.timezone.utc), 'volume_glance_metadata': [{'key': 'signature_verified', 'value': 'False'}], 'replication_status': None, 'service_uuid': None, 'provider_id': None, 'shared_targets': True, 'volume_type_id': '68adf31b-ae02-4f64-af1d-7071badf221b', 'size': 1, 'user_id': '2aa4f468a15546eb9f72cb2b632e67bb', 'display_name': 'image-90f3f35e-76ec-4c31-b581-329b315fe607', 'provider_location': None, 'launched_at': None, 'display_description': None, 'volume_metadata': [], 'project_id': 'b7875946723844198353bbfcf22516fb', 'host': 'ncdv-master-1@ceph#ceph', 'consistencygroup_id': None, 'attach_status': 'detached', 'status': 'creating', 'source_volid': 'ea35f176-da9d-49b3-9ff6-4dce9e05b3c8'}. {{(pid=133391) _clone_image_volume /opt/stack/cinder/cinder/volume/manager.py:1660}} Oct 01 20:51:51.155742 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume reschedule parameters: False retry: None {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 20:51:51.197282 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Flow 'volume_create_manager' (824ece14-5286-4396-b9ee-41e172c57fce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:51:51.206323 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (810869c5-4df2-40c7-bfec-1f49edd7616f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:51.258062 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (810869c5-4df2-40c7-bfec-1f49edd7616f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T20:51:04Z,deleted=False,deleted_at=None,display_description=None,display_name='image-90f3f35e-76ec-4c31-b581-329b315fe607',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=822f6692-9e60-4c53-96e6-c1632fc60252,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b7875946723844198353bbfcf22516fb',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T20:51:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=ea35f176-da9d-49b3-9ff6-4dce9e05b3c8,status='creating',terminated_at=None,updated_at=2022-10-01T20:51:05Z,use_quota=True,user_id='2aa4f468a15546eb9f72cb2b632e67bb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:51.277045 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ed0f9688-7bc4-4a5b-a1da-36279dbb2df3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:51.287436 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ed0f9688-7bc4-4a5b-a1da-36279dbb2df3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:51.301639 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (afd57926-15a1-4731-83bd-a49226b0a438) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:51.335801 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (afd57926-15a1-4731-83bd-a49226b0a438) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'volume_name': 'volume-822f6692-9e60-4c53-96e6-c1632fc60252', 'volume_size': 1, 'source_volid': 'ea35f176-da9d-49b3-9ff6-4dce9e05b3c8', 'source_volstatus': 'downloading'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:51.343297 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (658c2e31-a2f1-4c96-a74b-dfd9b84861d3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:51.350712 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (658c2e31-a2f1-4c96-a74b-dfd9b84861d3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:51.356216 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (322f1508-c3a3-4ca6-b2df-fe4ac74a8a75) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:51.357694 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume 822f6692-9e60-4c53-96e6-c1632fc60252: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-822f6692-9e60-4c53-96e6-c1632fc60252', 'volume_size': 1, 'source_volid': 'ea35f176-da9d-49b3-9ff6-4dce9e05b3c8', 'source_volstatus': 'downloading'} Oct 01 20:51:51.375599 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:51:51.476236 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] creating snapshot='volume-822f6692-9e60-4c53-96e6-c1632fc60252.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 20:51:52.259927 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] cloning 'volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8@volume-822f6692-9e60-4c53-96e6-c1632fc60252.clone_snap' to 'volume-822f6692-9e60-4c53-96e6-c1632fc60252' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 20:51:52.407858 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 20:51:52.409625 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 20:51:52.410655 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 20:51:52.444163 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 20:51:52.448405 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (322f1508-c3a3-4ca6-b2df-fe4ac74a8a75) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-822f6692-9e60-4c53-96e6-c1632fc60252', 'volume_size': 1, 'source_volid': 'ea35f176-da9d-49b3-9ff6-4dce9e05b3c8', 'source_volstatus': 'downloading'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:52.463414 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4f4d1274-2241-43f0-a869-3470a39242e4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:52.479841 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume volume-822f6692-9e60-4c53-96e6-c1632fc60252 (822f6692-9e60-4c53-96e6-c1632fc60252): created successfully Oct 01 20:51:52.484110 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4f4d1274-2241-43f0-a869-3470a39242e4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:52.498565 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Flow 'volume_create_manager' (824ece14-5286-4396-b9ee-41e172c57fce) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:51:52.512071 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Created volume successfully. Oct 01 20:51:52.560476 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Creating new image-volume cache entry for image 90f3f35e-76ec-4c31-b581-329b315fe607 on ncdv-master-1@ceph#ceph {{(pid=133391) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:104}} Oct 01 20:51:52.570507 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] New image-volume cache entry created: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 20, 51, 52, 563962)}. {{(pid=133391) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:129}} Oct 01 20:51:52.578226 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 47.250s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 20:51:52.578806 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Marking volume ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 20:51:52.592068 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to ea35f176-da9d-49b3-9ff6-4dce9e05b3c8. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 20:51:52.594027 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Creating volume glance metadata for volume ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 20:51:52.674866 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4dc395b6-eb3c-435d-aa67-74e4553bcc94) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:52.681505 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (784dc1cb-a41e-42ab-b5a0-8723161fa075) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:52.695779 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 (ea35f176-da9d-49b3-9ff6-4dce9e05b3c8): created successfully Oct 01 20:51:52.700520 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (784dc1cb-a41e-42ab-b5a0-8723161fa075) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:52.707557 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Flow 'volume_create_manager' (35e8156e-5ca0-4b23-86dd-11ee83ea8090) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:51:52.721291 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1fbcd0c2-d67f-4dad-9f88-a1a0ac23bc43 tempest-TaggedBootDevicesTest_v242-711726808 None] Created volume successfully. Oct 01 20:51:53.907188 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 20:51:53.927824 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Flow 'volume_create_manager' (e3038420-fbcd-4a2b-8ac5-66b7f1c788aa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:51:53.933931 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (601ac95f-1083-4eb3-9eef-d583f223bdeb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:53.951185 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (601ac95f-1083-4eb3-9eef-d583f223bdeb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T20:51:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-1445997529',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e3be1f38-f8d0-4c82-a901-0c7a1abac289,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='bb36a4c325b54c8a964113ba5e52f47b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T20:51:54Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T20:51:54Z,use_quota=True,user_id='2aa4f468a15546eb9f72cb2b632e67bb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:53.956406 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2c5ebba2-c85f-4b8c-bb85-c98bfe179615) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:53.961015 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2c5ebba2-c85f-4b8c-bb85-c98bfe179615) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:53.965923 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4b7a8c5d-0214-4b53-a1cf-d36410b312f8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:53.969782 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4b7a8c5d-0214-4b53-a1cf-d36410b312f8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e3be1f38-f8d0-4c82-a901-0c7a1abac289', 'volume_name': 'volume-e3be1f38-f8d0-4c82-a901-0c7a1abac289', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:53.975206 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (450855fb-b91c-42e2-998c-998f8418eaec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:53.979428 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (450855fb-b91c-42e2-998c-998f8418eaec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:53.984318 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (45b3e87a-4d3a-4715-94d6-30e3e62ae031) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:53.985451 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume e3be1f38-f8d0-4c82-a901-0c7a1abac289: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e3be1f38-f8d0-4c82-a901-0c7a1abac289', 'volume_size': 1} Oct 01 20:51:53.985862 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] creating volume 'volume-e3be1f38-f8d0-4c82-a901-0c7a1abac289' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 20:51:53.986694 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:51:54.070106 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 20:51:54.070869 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 20:51:54.074654 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (45b3e87a-4d3a-4715-94d6-30e3e62ae031) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e3be1f38-f8d0-4c82-a901-0c7a1abac289', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:54.079180 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6ba1fbbf-421e-45ad-89fe-b4785dd018bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:54.088504 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume volume-e3be1f38-f8d0-4c82-a901-0c7a1abac289 (e3be1f38-f8d0-4c82-a901-0c7a1abac289): created successfully Oct 01 20:51:54.092403 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6ba1fbbf-421e-45ad-89fe-b4785dd018bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:54.098099 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Flow 'volume_create_manager' (e3038420-fbcd-4a2b-8ac5-66b7f1c788aa) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:51:54.108169 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-cf2c19db-0e4c-4c7e-a817-d18a91d55484 tempest-TaggedBootDevicesTest_v242-711726808 None] Created volume successfully. Oct 01 20:51:55.447040 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 20:51:55.487591 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Flow 'volume_create_manager' (c8bdfb37-027f-4f9c-a112-d6c90afb6cb2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:51:55.509465 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (894bd93e-b7ac-42ef-a220-0ad8d4d36d15) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:55.535230 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (894bd93e-b7ac-42ef-a220-0ad8d4d36d15) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T20:51:55Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-904250802',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d4764752-297f-4f5f-bd4a-009489d9151b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='bb36a4c325b54c8a964113ba5e52f47b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T20:51:55Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T20:51:55Z,use_quota=True,user_id='2aa4f468a15546eb9f72cb2b632e67bb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:55.543666 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3f79c01d-a235-44d4-903e-ffd0bcfe456e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:55.552374 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3f79c01d-a235-44d4-903e-ffd0bcfe456e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:55.564967 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f0343bf3-c516-4005-a450-baa58b279254) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:55.570951 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f0343bf3-c516-4005-a450-baa58b279254) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd4764752-297f-4f5f-bd4a-009489d9151b', 'volume_name': 'volume-d4764752-297f-4f5f-bd4a-009489d9151b', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:55.580490 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (26c001b3-3d03-4c4c-8302-845fc451b05d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:55.587155 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (26c001b3-3d03-4c4c-8302-845fc451b05d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:55.594974 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (eb438193-7aca-46df-acb7-bd77f5b08cfd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:55.596938 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume d4764752-297f-4f5f-bd4a-009489d9151b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d4764752-297f-4f5f-bd4a-009489d9151b', 'volume_size': 1} Oct 01 20:51:55.598123 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] creating volume 'volume-d4764752-297f-4f5f-bd4a-009489d9151b' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 20:51:55.599212 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:51:55.746988 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 20:51:55.747706 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 20:51:55.751579 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (eb438193-7aca-46df-acb7-bd77f5b08cfd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d4764752-297f-4f5f-bd4a-009489d9151b', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:55.757144 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bdbce201-158d-4bde-ad7b-360d5dfcb795) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:51:55.793449 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume volume-d4764752-297f-4f5f-bd4a-009489d9151b (d4764752-297f-4f5f-bd4a-009489d9151b): created successfully Oct 01 20:51:55.802629 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bdbce201-158d-4bde-ad7b-360d5dfcb795) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:51:55.809593 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Flow 'volume_create_manager' (c8bdfb37-027f-4f9c-a112-d6c90afb6cb2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:51:55.821927 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b54806d2-b863-47a9-8aca-449de0144061 tempest-TaggedBootDevicesTest_v242-711726808 None] Created volume successfully. Oct 01 20:52:05.827940 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=14,cinder:SELECT=49,cinder:INSERT=13,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:18.014282 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-b336db30-0011-4a45-9936-67cb95be2f96 tempest-TaggedBootDevicesTest_v242-711726808 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:52:19.502736 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-b336db30-0011-4a45-9936-67cb95be2f96 tempest-TaggedBootDevicesTest_v242-711726808 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.488s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:52:19.503397 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-b336db30-0011-4a45-9936-67cb95be2f96 tempest-TaggedBootDevicesTest_v242-711726808 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'ea35f176-da9d-49b3-9ff6-4dce9e05b3c8', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 20:52:19.587685 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-b336db30-0011-4a45-9936-67cb95be2f96 tempest-TaggedBootDevicesTest_v242-711726808 None] Connection info returned from driver {'name': 'volumes/volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'ea35f176-da9d-49b3-9ff6-4dce9e05b3c8', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'd50a1401-4d34-4241-a195-ce773ee990c1'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 20:52:19.674975 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-b336db30-0011-4a45-9936-67cb95be2f96 tempest-TaggedBootDevicesTest_v242-711726808 None] attachment_update completed successfully. Oct 01 20:52:20.167857 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-77c37330-3bd7-43ef-a984-11a09dc57cbc tempest-TaggedBootDevicesTest_v242-711726808 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:52:21.325124 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-77c37330-3bd7-43ef-a984-11a09dc57cbc tempest-TaggedBootDevicesTest_v242-711726808 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.157s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:52:21.326470 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-77c37330-3bd7-43ef-a984-11a09dc57cbc tempest-TaggedBootDevicesTest_v242-711726808 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e3be1f38-f8d0-4c82-a901-0c7a1abac289', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'e3be1f38-f8d0-4c82-a901-0c7a1abac289', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 20:52:21.340508 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=53,cinder:INSERT=11,cinder:UPDATE=15,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:21.358685 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-77c37330-3bd7-43ef-a984-11a09dc57cbc tempest-TaggedBootDevicesTest_v242-711726808 None] Connection info returned from driver {'name': 'volumes/volume-e3be1f38-f8d0-4c82-a901-0c7a1abac289', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'e3be1f38-f8d0-4c82-a901-0c7a1abac289', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '26aacba7-4bbb-468a-9e74-7cb9fa616eb1'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 20:52:21.434665 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-77c37330-3bd7-43ef-a984-11a09dc57cbc tempest-TaggedBootDevicesTest_v242-711726808 None] attachment_update completed successfully. Oct 01 20:52:21.870939 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-c10ff0f0-8af1-4f6c-937c-a2b0e7347d1c tempest-TaggedBootDevicesTest_v242-711726808 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:52:23.003982 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-c10ff0f0-8af1-4f6c-937c-a2b0e7347d1c tempest-TaggedBootDevicesTest_v242-711726808 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.133s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:52:23.004747 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-c10ff0f0-8af1-4f6c-937c-a2b0e7347d1c tempest-TaggedBootDevicesTest_v242-711726808 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-d4764752-297f-4f5f-bd4a-009489d9151b', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'd4764752-297f-4f5f-bd4a-009489d9151b', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 20:52:23.040846 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-c10ff0f0-8af1-4f6c-937c-a2b0e7347d1c tempest-TaggedBootDevicesTest_v242-711726808 None] Connection info returned from driver {'name': 'volumes/volume-d4764752-297f-4f5f-bd4a-009489d9151b', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'd4764752-297f-4f5f-bd4a-009489d9151b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '54b22890-654f-40e9-95ac-8e274986595a'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 20:52:23.115098 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-1b85b5b4-d9fe-4fb1-ac73-5f732e43a16e req-c10ff0f0-8af1-4f6c-937c-a2b0e7347d1c tempest-TaggedBootDevicesTest_v242-711726808 None] attachment_update completed successfully. Oct 01 20:52:23.701510 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:52:23.703600 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:52:23.783445 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:52:23.900339 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:52:42.837133 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=43,cinder:UPDATE=8 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:52:52.843189 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=20,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:19.526103 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 20:53:19.548493 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Flow 'volume_create_manager' (3092e15d-0abe-418b-b8d1-9790fcc241c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:53:19.555045 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (afb67723-0715-4f0a-b3b2-06db610eefb1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:53:19.559633 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:19.573170 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (afb67723-0715-4f0a-b3b2-06db610eefb1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T20:53:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveAutoBlockMigrationV225Test-volume-292438540',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2ffa640e-11d3-4b20-a2d6-2b0316d715b0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='76e0a9843d9c41f7b2fae591dc09d9d2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T20:53:19Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T20:53:19Z,use_quota=True,user_id='aa72870453cd4b958725801b842c7a28',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:53:19.579168 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9a81265a-a6fe-43d5-bef8-2b72be2d2e47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:53:19.583341 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9a81265a-a6fe-43d5-bef8-2b72be2d2e47) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:53:19.589162 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (da1b3c5d-96a3-459c-86bb-5ea29c3dac8a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:53:19.593465 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (da1b3c5d-96a3-459c-86bb-5ea29c3dac8a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'volume_name': 'volume-2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:53:19.601147 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3f437ae3-1bc7-4267-a73f-80fdbf0b8e01) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:53:19.605999 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3f437ae3-1bc7-4267-a73f-80fdbf0b8e01) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:53:19.614013 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ff497dc3-bdda-47bc-a152-4ca98445345a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:53:19.615317 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Volume 2ffa640e-11d3-4b20-a2d6-2b0316d715b0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'volume_size': 1} Oct 01 20:53:19.615877 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] creating volume 'volume-2ffa640e-11d3-4b20-a2d6-2b0316d715b0' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 20:53:19.616728 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:53:19.755275 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 20:53:19.756037 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 20:53:19.763027 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ff497dc3-bdda-47bc-a152-4ca98445345a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:53:19.771993 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6159c7ff-8791-48cf-adb7-d69376d33ac4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:53:19.783996 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Volume volume-2ffa640e-11d3-4b20-a2d6-2b0316d715b0 (2ffa640e-11d3-4b20-a2d6-2b0316d715b0): created successfully Oct 01 20:53:19.790322 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6159c7ff-8791-48cf-adb7-d69376d33ac4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:53:19.800573 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Flow 'volume_create_manager' (3092e15d-0abe-418b-b8d1-9790fcc241c3) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:53:19.813807 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5e7747f0-5c2d-4fd0-959e-4884350eb632 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Created volume successfully. Oct 01 20:53:23.498866 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=11,cinder:UPDATE=3,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:53:23.511003 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-b120f13f-7129-4b56-8000-94c52ca2a2dc req-c242beb5-ccdb-4d6d-bcb6-d08ca09edafe tempest-LiveAutoBlockMigrationV225Test-121044104 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:53:23.707426 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:53:23.709319 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:53:23.748326 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:53:23.798266 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:53:24.334612 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-b120f13f-7129-4b56-8000-94c52ca2a2dc req-c242beb5-ccdb-4d6d-bcb6-d08ca09edafe tempest-LiveAutoBlockMigrationV225Test-121044104 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.824s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:53:24.335596 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-b120f13f-7129-4b56-8000-94c52ca2a2dc req-c242beb5-ccdb-4d6d-bcb6-d08ca09edafe tempest-LiveAutoBlockMigrationV225Test-121044104 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 20:53:24.388118 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-b120f13f-7129-4b56-8000-94c52ca2a2dc req-c242beb5-ccdb-4d6d-bcb6-d08ca09edafe tempest-LiveAutoBlockMigrationV225Test-121044104 None] Connection info returned from driver {'name': 'volumes/volume-2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '6be733d2-9c82-47b3-9df7-82c23a72cb20'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 20:53:24.454855 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-b120f13f-7129-4b56-8000-94c52ca2a2dc req-c242beb5-ccdb-4d6d-bcb6-d08ca09edafe tempest-LiveAutoBlockMigrationV225Test-121044104 None] attachment_update completed successfully. Oct 01 20:53:39.376171 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-03379b7b-bc83-4307-baa9-0a944f396893 req-2a45a5e2-6857-4184-bfe2-cbc12a2b9eee tempest-LiveAutoBlockMigrationV225Test-1233515141 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:53:40.238564 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-03379b7b-bc83-4307-baa9-0a944f396893 req-2a45a5e2-6857-4184-bfe2-cbc12a2b9eee tempest-LiveAutoBlockMigrationV225Test-1233515141 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.861s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:53:40.241143 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-03379b7b-bc83-4307-baa9-0a944f396893 req-2a45a5e2-6857-4184-bfe2-cbc12a2b9eee tempest-LiveAutoBlockMigrationV225Test-1233515141 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 20:53:40.271344 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-03379b7b-bc83-4307-baa9-0a944f396893 req-2a45a5e2-6857-4184-bfe2-cbc12a2b9eee tempest-LiveAutoBlockMigrationV225Test-1233515141 None] Connection info returned from driver {'name': 'volumes/volume-2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '2ffa640e-11d3-4b20-a2d6-2b0316d715b0', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'cdc2654f-b172-4310-9fd5-52a01bcffa8a'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 20:53:40.334452 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-03379b7b-bc83-4307-baa9-0a944f396893 req-2a45a5e2-6857-4184-bfe2-cbc12a2b9eee tempest-LiveAutoBlockMigrationV225Test-1233515141 None] attachment_update completed successfully. Oct 01 20:53:49.598065 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-03379b7b-bc83-4307-baa9-0a944f396893 req-c3f0ebd7-52d7-4f51-bee8-790cfc6c6856 tempest-LiveAutoBlockMigrationV225Test-1233515141 None] Terminate volume connection completed successfully. Oct 01 20:53:49.598695 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-03379b7b-bc83-4307-baa9-0a944f396893 req-c3f0ebd7-52d7-4f51-bee8-790cfc6c6856 tempest-LiveAutoBlockMigrationV225Test-1233515141 None] Deleting attachment 6be733d2-9c82-47b3-9df7-82c23a72cb20. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 20:53:52.831667 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=30,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:54:02.831341 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=31,cinder:UPDATE=8 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:54:06.309294 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-c6e72308-3905-4b92-9903-1ada707a39db req-14ba7912-08fa-47ea-8335-1492f8dabfe2 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Terminate volume connection completed successfully. Oct 01 20:54:06.309823 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-c6e72308-3905-4b92-9903-1ada707a39db req-14ba7912-08fa-47ea-8335-1492f8dabfe2 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Deleting attachment cdc2654f-b172-4310-9fd5-52a01bcffa8a. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 20:54:07.779842 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 20:54:07.807837 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Flow 'volume_create_manager' (a46706fe-9deb-4bbb-96e3-abbf380cf567) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:54:07.815359 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7cd695e7-79f4-439e-a455-560fbaf828ed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:07.833882 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7cd695e7-79f4-439e-a455-560fbaf828ed) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T20:54:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1825954236',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=0657d824-3635-4137-a555-f2ea7736e55a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='76e0a9843d9c41f7b2fae591dc09d9d2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T20:54:08Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T20:54:08Z,use_quota=True,user_id='aa72870453cd4b958725801b842c7a28',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:07.839764 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51688119-5c04-4b26-9e79-255a75589130) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:07.845856 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51688119-5c04-4b26-9e79-255a75589130) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:07.853558 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d51f35d4-7d87-4b33-9998-816b9df553f1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:08.012972 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d51f35d4-7d87-4b33-9998-816b9df553f1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '0657d824-3635-4137-a555-f2ea7736e55a', 'volume_name': 'volume-0657d824-3635-4137-a555-f2ea7736e55a', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:08.026753 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f467e052-0e94-46af-99f7-6da5ed02f474) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:08.040901 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f467e052-0e94-46af-99f7-6da5ed02f474) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:08.059154 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3b1db670-ccf0-427a-bdea-f570cb5f5f09) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:08.061237 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Volume 0657d824-3635-4137-a555-f2ea7736e55a: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-0657d824-3635-4137-a555-f2ea7736e55a', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 20:54:08.061933 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Cloning 0657d824-3635-4137-a555-f2ea7736e55a from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 20:54:08.064565 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 20:54:08.087394 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.022s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 20:54:08.121913 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 20, 54, 8, 104179)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 20:54:08.122362 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 20:54:08.122845 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 20:54:08.133754 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.047s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 20:54:08.134934 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 20:54:08.154703 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 20, 54, 8, 145023)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 20:54:08.155413 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 20:54:08.155987 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 20:54:08.198901 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:54:08.335960 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] creating snapshot='volume-0657d824-3635-4137-a555-f2ea7736e55a.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 20:54:09.320115 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-0657d824-3635-4137-a555-f2ea7736e55a.clone_snap' to 'volume-0657d824-3635-4137-a555-f2ea7736e55a' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 20:54:09.569553 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 20:54:09.570754 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 20:54:09.571138 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 20:54:09.651748 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 20:54:09.652928 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Marking volume 0657d824-3635-4137-a555-f2ea7736e55a as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 20:54:09.677099 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to 0657d824-3635-4137-a555-f2ea7736e55a. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 20:54:09.678703 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Creating volume glance metadata for volume 0657d824-3635-4137-a555-f2ea7736e55a backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 20:54:09.841645 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3b1db670-ccf0-427a-bdea-f570cb5f5f09) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-0657d824-3635-4137-a555-f2ea7736e55a', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:09.849217 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e2b30bdf-bab7-4fb6-b52c-e1dd05fcdb08) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:09.869721 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Volume volume-0657d824-3635-4137-a555-f2ea7736e55a (0657d824-3635-4137-a555-f2ea7736e55a): created successfully Oct 01 20:54:09.878665 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e2b30bdf-bab7-4fb6-b52c-e1dd05fcdb08) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:09.893542 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Flow 'volume_create_manager' (a46706fe-9deb-4bbb-96e3-abbf380cf567) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:54:09.906634 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d8003f92-d8ad-4d42-84c3-03936a4a5184 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Created volume successfully. Oct 01 20:54:15.107469 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-944608fe-c172-45f5-a8aa-57b70be639bd req-97869acb-dfdd-45c5-b207-c9548ca6e396 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:54:15.902838 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-944608fe-c172-45f5-a8aa-57b70be639bd req-97869acb-dfdd-45c5-b207-c9548ca6e396 tempest-LiveAutoBlockMigrationV225Test-121044104 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.795s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:54:15.904271 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-944608fe-c172-45f5-a8aa-57b70be639bd req-97869acb-dfdd-45c5-b207-c9548ca6e396 tempest-LiveAutoBlockMigrationV225Test-121044104 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-0657d824-3635-4137-a555-f2ea7736e55a', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '0657d824-3635-4137-a555-f2ea7736e55a', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 20:54:15.928802 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-944608fe-c172-45f5-a8aa-57b70be639bd req-97869acb-dfdd-45c5-b207-c9548ca6e396 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Connection info returned from driver {'name': 'volumes/volume-0657d824-3635-4137-a555-f2ea7736e55a', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '0657d824-3635-4137-a555-f2ea7736e55a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '844b6970-f231-40e4-acba-3aa145596621'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 20:54:15.982353 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-944608fe-c172-45f5-a8aa-57b70be639bd req-97869acb-dfdd-45c5-b207-c9548ca6e396 tempest-LiveAutoBlockMigrationV225Test-121044104 None] attachment_update completed successfully. Oct 01 20:54:23.705846 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:54:23.707716 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:54:23.741785 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:54:23.777562 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:54:25.982823 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=34,cinder:UPDATE=4,cinder:INSERT=6 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:54:32.165388 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=30,cinder:UPDATE=8,cinder:INSERT=6,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:54:32.192948 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 20:54:32.216602 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Flow 'volume_create_manager' (b188354d-4c92-46df-ade4-65b8fb9c9fdf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:54:32.223371 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5739aa35-50ca-47b3-81fd-3aeb023f730c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:32.242298 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5739aa35-50ca-47b3-81fd-3aeb023f730c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T20:54:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveMigrationTest-volume-1746768173',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f83c038c-0838-41d4-b421-20d53a0dec11,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='237c6bad691343e7a7e64304f55e01e9',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T20:54:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T20:54:32Z,use_quota=True,user_id='a749a74841e840e2b8de67fb9562bf51',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:32.248662 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c68bdb74-1134-44d3-a70a-9ea0c20321b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:32.254703 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c68bdb74-1134-44d3-a70a-9ea0c20321b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:32.260112 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (319e3cf8-a7b6-4b82-8978-1756d92b7d49) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:32.265408 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (319e3cf8-a7b6-4b82-8978-1756d92b7d49) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f83c038c-0838-41d4-b421-20d53a0dec11', 'volume_name': 'volume-f83c038c-0838-41d4-b421-20d53a0dec11', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:32.271087 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f8ad64bf-2e77-465d-a82d-a2ff99b8aa96) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:32.275701 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f8ad64bf-2e77-465d-a82d-a2ff99b8aa96) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:32.281618 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d6cd196b-20eb-4b52-ad8e-699f7df1b963) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:32.282762 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Volume f83c038c-0838-41d4-b421-20d53a0dec11: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f83c038c-0838-41d4-b421-20d53a0dec11', 'volume_size': 1} Oct 01 20:54:32.283405 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] creating volume 'volume-f83c038c-0838-41d4-b421-20d53a0dec11' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 20:54:32.284435 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:54:32.371689 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 20:54:32.372070 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 20:54:32.376856 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d6cd196b-20eb-4b52-ad8e-699f7df1b963) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f83c038c-0838-41d4-b421-20d53a0dec11', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:32.382215 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (aec46096-42bb-44fb-b6a7-bb50491f2ff3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:54:32.393316 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Volume volume-f83c038c-0838-41d4-b421-20d53a0dec11 (f83c038c-0838-41d4-b421-20d53a0dec11): created successfully Oct 01 20:54:32.396910 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (aec46096-42bb-44fb-b6a7-bb50491f2ff3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:54:32.402898 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Flow 'volume_create_manager' (b188354d-4c92-46df-ade4-65b8fb9c9fdf) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:54:32.413183 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-98f7ca71-b0b5-425a-8e3b-78f93b70a0f0 tempest-LiveMigrationTest-1978574502 None] Created volume successfully. Oct 01 20:54:35.290530 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-37fcb9fc-58e1-47e7-ad37-fe3ce22b082b req-c53fa2ac-a6a7-45d0-93de-fda8647ff996 tempest-LiveMigrationTest-1978574502 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:54:36.408168 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-37fcb9fc-58e1-47e7-ad37-fe3ce22b082b req-c53fa2ac-a6a7-45d0-93de-fda8647ff996 tempest-LiveMigrationTest-1978574502 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.117s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:54:36.410701 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-37fcb9fc-58e1-47e7-ad37-fe3ce22b082b req-c53fa2ac-a6a7-45d0-93de-fda8647ff996 tempest-LiveMigrationTest-1978574502 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f83c038c-0838-41d4-b421-20d53a0dec11', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'f83c038c-0838-41d4-b421-20d53a0dec11', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 20:54:36.444283 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-37fcb9fc-58e1-47e7-ad37-fe3ce22b082b req-c53fa2ac-a6a7-45d0-93de-fda8647ff996 tempest-LiveMigrationTest-1978574502 None] Connection info returned from driver {'name': 'volumes/volume-f83c038c-0838-41d4-b421-20d53a0dec11', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'f83c038c-0838-41d4-b421-20d53a0dec11', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '0afc0692-667f-4287-b147-4d936f2a2d0c'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 20:54:36.538878 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-37fcb9fc-58e1-47e7-ad37-fe3ce22b082b req-c53fa2ac-a6a7-45d0-93de-fda8647ff996 tempest-LiveMigrationTest-1978574502 None] attachment_update completed successfully. Oct 01 20:54:42.450603 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-5e64de91-823a-47b5-9ad2-f931b830dbb3 req-476bd33b-5fe6-486d-8eed-5247de23c50d tempest-LiveAutoBlockMigrationV225Test-1233515141 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:54:43.514866 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-5e64de91-823a-47b5-9ad2-f931b830dbb3 req-476bd33b-5fe6-486d-8eed-5247de23c50d tempest-LiveAutoBlockMigrationV225Test-1233515141 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.063s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:54:43.514866 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-5e64de91-823a-47b5-9ad2-f931b830dbb3 req-476bd33b-5fe6-486d-8eed-5247de23c50d tempest-LiveAutoBlockMigrationV225Test-1233515141 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-0657d824-3635-4137-a555-f2ea7736e55a', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '0657d824-3635-4137-a555-f2ea7736e55a', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 20:54:43.596052 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-5e64de91-823a-47b5-9ad2-f931b830dbb3 req-476bd33b-5fe6-486d-8eed-5247de23c50d tempest-LiveAutoBlockMigrationV225Test-1233515141 None] Connection info returned from driver {'name': 'volumes/volume-0657d824-3635-4137-a555-f2ea7736e55a', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '0657d824-3635-4137-a555-f2ea7736e55a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '016a1386-c977-417f-9e00-03e7a2609859'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 20:54:43.674947 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-5e64de91-823a-47b5-9ad2-f931b830dbb3 req-476bd33b-5fe6-486d-8eed-5247de23c50d tempest-LiveAutoBlockMigrationV225Test-1233515141 None] attachment_update completed successfully. Oct 01 20:54:46.802542 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-05b62cd0-eccb-48ca-9323-4eafc6edbbf0 req-103fd01e-3cab-4f25-81a6-3578d6c5eb03 tempest-LiveMigrationTest-894622269 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:54:47.780754 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-05b62cd0-eccb-48ca-9323-4eafc6edbbf0 req-103fd01e-3cab-4f25-81a6-3578d6c5eb03 tempest-LiveMigrationTest-894622269 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.978s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:54:47.781932 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-05b62cd0-eccb-48ca-9323-4eafc6edbbf0 req-103fd01e-3cab-4f25-81a6-3578d6c5eb03 tempest-LiveMigrationTest-894622269 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f83c038c-0838-41d4-b421-20d53a0dec11', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'f83c038c-0838-41d4-b421-20d53a0dec11', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 20:54:47.817113 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-05b62cd0-eccb-48ca-9323-4eafc6edbbf0 req-103fd01e-3cab-4f25-81a6-3578d6c5eb03 tempest-LiveMigrationTest-894622269 None] Connection info returned from driver {'name': 'volumes/volume-f83c038c-0838-41d4-b421-20d53a0dec11', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'f83c038c-0838-41d4-b421-20d53a0dec11', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '759c16b0-d1c8-4f75-a9c0-80aecb917df9'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 20:54:47.887108 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-05b62cd0-eccb-48ca-9323-4eafc6edbbf0 req-103fd01e-3cab-4f25-81a6-3578d6c5eb03 tempest-LiveMigrationTest-894622269 None] attachment_update completed successfully. Oct 01 20:54:53.819850 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-5e64de91-823a-47b5-9ad2-f931b830dbb3 req-6b591916-e28f-4d87-8b1e-db6e9e9f8596 tempest-LiveAutoBlockMigrationV225Test-1233515141 None] Terminate volume connection completed successfully. Oct 01 20:54:53.821426 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-5e64de91-823a-47b5-9ad2-f931b830dbb3 req-6b591916-e28f-4d87-8b1e-db6e9e9f8596 tempest-LiveAutoBlockMigrationV225Test-1233515141 None] Deleting attachment 844b6970-f231-40e4-acba-3aa145596621. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 20:55:02.329715 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=47,cinder:UPDATE=3,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:02.332627 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=42,cinder:UPDATE=12 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:02.373528 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-05b62cd0-eccb-48ca-9323-4eafc6edbbf0 req-e71bd39c-6dbd-4ef8-b48c-a3b7fbb1f708 tempest-LiveMigrationTest-894622269 None] Terminate volume connection completed successfully. Oct 01 20:55:02.377512 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-05b62cd0-eccb-48ca-9323-4eafc6edbbf0 req-e71bd39c-6dbd-4ef8-b48c-a3b7fbb1f708 tempest-LiveMigrationTest-894622269 None] Deleting attachment 0afc0692-667f-4287-b147-4d936f2a2d0c. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 20:55:08.966658 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-1ea0e3ec-be3c-4155-a6cf-b91ee7e7af45 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Terminate volume connection completed successfully. Oct 01 20:55:08.967706 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-1ea0e3ec-be3c-4155-a6cf-b91ee7e7af45 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Deleting attachment 016a1386-c977-417f-9e00-03e7a2609859. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 20:55:09.227023 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Acquiring lock "cinder-0657d824-3635-4137-a555-f2ea7736e55a-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 20:55:09.241293 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Lock "cinder-0657d824-3635-4137-a555-f2ea7736e55a-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 20:55:09.285536 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:55:09.392467 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 20:55:09.404819 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] deleting rbd volume volume-0657d824-3635-4137-a555-f2ea7736e55a {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 20:55:09.601255 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 20:55:09.628991 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] deleting parent snapshot volume-0657d824-3635-4137-a555-f2ea7736e55a.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 20:55:10.279895 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Created reservations ['922d533c-d6d2-4927-a15d-ceba233781f8', 'e9a5aba6-3a4d-455f-98e4-2c959e081505', 'e30d3a82-b237-4ca7-a142-4ddd41be1403', 'c4146979-ab34-40bb-89a9-88b00075e939'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 20:55:10.368184 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:55:10.444384 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:55:10.502100 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:55:10.508606 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Deleted volume successfully. Oct 01 20:55:10.531815 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-6ad507b2-1c82-4038-96fa-1eba25fbecc6 req-e506a7c4-6bf8-42da-a5cc-ab3aa6558fd5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Lock "cinder-0657d824-3635-4137-a555-f2ea7736e55a-delete_volume" released by "delete_volume" :: held 1.291s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 20:55:11.547265 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Acquiring lock "cinder-2ffa640e-11d3-4b20-a2d6-2b0316d715b0-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 20:55:11.574478 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Lock "cinder-2ffa640e-11d3-4b20-a2d6-2b0316d715b0-delete_volume" acquired by "delete_volume" :: waited 0.027s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 20:55:11.650358 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:55:11.717802 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 20:55:11.718206 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Volume volume-2ffa640e-11d3-4b20-a2d6-2b0316d715b0 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 20:55:11.726433 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] deleting rbd volume volume-2ffa640e-11d3-4b20-a2d6-2b0316d715b0 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 20:55:12.122929 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Created reservations ['f9622379-ae9f-422d-ab8a-81e1d2f6be75', 'cb3534bb-a43c-4a00-a215-e00414c2f33d', 'f7ca9837-fb67-4f4d-9ecf-77cc99b4acb4', '99e94fdb-0125-4d6c-8465-6d3f07df72b8'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 20:55:12.231046 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:55:12.326133 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:55:12.391796 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:55:12.397872 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Deleted volume successfully. Oct 01 20:55:12.426007 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7afe5089-467c-4fff-99ba-4254c901d6c5 tempest-LiveAutoBlockMigrationV225Test-121044104 None] Lock "cinder-2ffa640e-11d3-4b20-a2d6-2b0316d715b0-delete_volume" released by "delete_volume" :: held 0.852s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 20:55:16.373417 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-21e96521-8837-4c40-8e1f-ec62547b9596 req-66b4f8cc-50fe-4236-a7f2-1d2199a177c8 tempest-LiveMigrationTest-1978574502 None] Terminate volume connection completed successfully. Oct 01 20:55:16.374551 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-21e96521-8837-4c40-8e1f-ec62547b9596 req-66b4f8cc-50fe-4236-a7f2-1d2199a177c8 tempest-LiveMigrationTest-1978574502 None] Deleting attachment 759c16b0-d1c8-4f75-a9c0-80aecb917df9. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 20:55:18.594324 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 20:55:18.631304 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Flow 'volume_create_manager' (5e929f75-dfca-40a1-b59d-688dbb383bad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:55:18.640066 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (33c70b0d-55bb-458c-951a-df993cb5434b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:55:18.667786 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (33c70b0d-55bb-458c-951a-df993cb5434b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T20:55:18Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1482811327',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bae2c97c-a30a-424b-bc18-c9977d768266,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='237c6bad691343e7a7e64304f55e01e9',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T20:55:19Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T20:55:19Z,use_quota=True,user_id='a749a74841e840e2b8de67fb9562bf51',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:55:18.673578 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5883a46d-1ea7-4bae-bbe4-7b6512b4729d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:55:18.678014 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5883a46d-1ea7-4bae-bbe4-7b6512b4729d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:55:18.684194 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2febf052-f891-4a5f-9a3b-5f1ad695a8ea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:55:18.856614 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2febf052-f891-4a5f-9a3b-5f1ad695a8ea) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'bae2c97c-a30a-424b-bc18-c9977d768266', 'volume_name': 'volume-bae2c97c-a30a-424b-bc18-c9977d768266', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:55:18.866065 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4d441129-ac58-4196-9de7-6072c5123cb3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:55:18.875037 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4d441129-ac58-4196-9de7-6072c5123cb3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:55:18.884499 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2115fbda-bb95-45b0-891a-b4cbb723b670) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:55:18.889284 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Volume bae2c97c-a30a-424b-bc18-c9977d768266: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-bae2c97c-a30a-424b-bc18-c9977d768266', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 20:55:18.890210 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Cloning bae2c97c-a30a-424b-bc18-c9977d768266 from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 20:55:18.891548 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 20:55:18.904610 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.013s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 20:55:18.933509 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 20, 55, 18, 914845)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 20:55:18.934714 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 20:55:18.935980 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 20:55:18.946680 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.042s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 20:55:18.949260 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 20:55:18.960845 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 20, 55, 18, 957597)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 20:55:18.961328 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 20:55:18.962174 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 20:55:18.984323 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:55:19.087697 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] creating snapshot='volume-bae2c97c-a30a-424b-bc18-c9977d768266.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 20:55:19.327417 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-bae2c97c-a30a-424b-bc18-c9977d768266.clone_snap' to 'volume-bae2c97c-a30a-424b-bc18-c9977d768266' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 20:55:19.509491 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 20:55:19.512605 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 20:55:19.513288 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 20:55:19.537421 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 20:55:19.538578 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Marking volume bae2c97c-a30a-424b-bc18-c9977d768266 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 20:55:19.549949 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to bae2c97c-a30a-424b-bc18-c9977d768266. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 20:55:19.551713 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Creating volume glance metadata for volume bae2c97c-a30a-424b-bc18-c9977d768266 backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 20:55:19.674413 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2115fbda-bb95-45b0-891a-b4cbb723b670) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bae2c97c-a30a-424b-bc18-c9977d768266', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:55:19.697374 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9c3e1e68-fd03-4a8a-8cd5-bdec2e9fd05f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 20:55:19.730460 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Volume volume-bae2c97c-a30a-424b-bc18-c9977d768266 (bae2c97c-a30a-424b-bc18-c9977d768266): created successfully Oct 01 20:55:19.734193 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9c3e1e68-fd03-4a8a-8cd5-bdec2e9fd05f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 20:55:19.751801 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Flow 'volume_create_manager' (5e929f75-dfca-40a1-b59d-688dbb383bad) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 20:55:19.768114 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ec4946cf-7bbc-4822-ace2-f845fcb54cdb tempest-LiveMigrationTest-1978574502 None] Created volume successfully. Oct 01 20:55:23.704011 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:55:23.704695 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:55:23.755604 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:55:23.818271 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:55:28.061485 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-a437d746-c0d7-49a8-8cb2-4547492dccd4 req-e6f6a69d-0945-43ed-9ef5-8df6f5b330f1 tempest-LiveMigrationTest-1978574502 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:55:29.164448 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-a437d746-c0d7-49a8-8cb2-4547492dccd4 req-e6f6a69d-0945-43ed-9ef5-8df6f5b330f1 tempest-LiveMigrationTest-1978574502 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.103s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:55:29.165989 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-a437d746-c0d7-49a8-8cb2-4547492dccd4 req-e6f6a69d-0945-43ed-9ef5-8df6f5b330f1 tempest-LiveMigrationTest-1978574502 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-bae2c97c-a30a-424b-bc18-c9977d768266', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'bae2c97c-a30a-424b-bc18-c9977d768266', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 20:55:29.210060 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-a437d746-c0d7-49a8-8cb2-4547492dccd4 req-e6f6a69d-0945-43ed-9ef5-8df6f5b330f1 tempest-LiveMigrationTest-1978574502 None] Connection info returned from driver {'name': 'volumes/volume-bae2c97c-a30a-424b-bc18-c9977d768266', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'bae2c97c-a30a-424b-bc18-c9977d768266', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'b375c178-434c-466c-950a-5d74d5c47c21'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 20:55:29.252662 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=57,cinder:UPDATE=29,cinder:INSERT=13,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:29.266225 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-a437d746-c0d7-49a8-8cb2-4547492dccd4 req-e6f6a69d-0945-43ed-9ef5-8df6f5b330f1 tempest-LiveMigrationTest-1978574502 None] attachment_update completed successfully. Oct 01 20:55:32.855023 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=62,cinder:UPDATE=15,cinder:INSERT=7,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:55:45.484082 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-20c55b8f-fef4-4f08-b9d4-d5c210c94ff5 req-d235236f-f972-4ea6-be47-1015e1905361 tempest-LiveMigrationTest-894622269 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 20:55:46.623734 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-20c55b8f-fef4-4f08-b9d4-d5c210c94ff5 req-d235236f-f972-4ea6-be47-1015e1905361 tempest-LiveMigrationTest-894622269 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.139s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 20:55:46.627227 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-20c55b8f-fef4-4f08-b9d4-d5c210c94ff5 req-d235236f-f972-4ea6-be47-1015e1905361 tempest-LiveMigrationTest-894622269 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-bae2c97c-a30a-424b-bc18-c9977d768266', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'bae2c97c-a30a-424b-bc18-c9977d768266', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 20:55:46.720493 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-20c55b8f-fef4-4f08-b9d4-d5c210c94ff5 req-d235236f-f972-4ea6-be47-1015e1905361 tempest-LiveMigrationTest-894622269 None] Connection info returned from driver {'name': 'volumes/volume-bae2c97c-a30a-424b-bc18-c9977d768266', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'bae2c97c-a30a-424b-bc18-c9977d768266', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '1e194d41-e9b7-404c-a9bb-96dbbf20963d'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 20:55:46.820193 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-20c55b8f-fef4-4f08-b9d4-d5c210c94ff5 req-d235236f-f972-4ea6-be47-1015e1905361 tempest-LiveMigrationTest-894622269 None] attachment_update completed successfully. Oct 01 20:55:56.819655 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=16,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:00.498899 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=16,cinder:UPDATE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:00.524510 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-20c55b8f-fef4-4f08-b9d4-d5c210c94ff5 req-f983a759-a07d-4ed1-95cb-1b1e1f7db8ad tempest-LiveMigrationTest-894622269 None] Terminate volume connection completed successfully. Oct 01 20:56:00.537255 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-20c55b8f-fef4-4f08-b9d4-d5c210c94ff5 req-f983a759-a07d-4ed1-95cb-1b1e1f7db8ad tempest-LiveMigrationTest-894622269 None] Deleting attachment b375c178-434c-466c-950a-5d74d5c47c21. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 20:56:08.390549 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-5b6f5a28-6322-49ce-8479-7f073b92a0f2 tempest-LiveMigrationTest-1978574502 None] Terminate volume connection completed successfully. Oct 01 20:56:08.391147 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-5b6f5a28-6322-49ce-8479-7f073b92a0f2 tempest-LiveMigrationTest-1978574502 None] Deleting attachment 1e194d41-e9b7-404c-a9bb-96dbbf20963d. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 20:56:08.615895 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] Acquiring lock "cinder-bae2c97c-a30a-424b-bc18-c9977d768266-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 20:56:08.633551 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] Lock "cinder-bae2c97c-a30a-424b-bc18-c9977d768266-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 20:56:08.690901 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:56:08.823510 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 20:56:08.849044 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] deleting rbd volume volume-bae2c97c-a30a-424b-bc18-c9977d768266 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 20:56:09.190684 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 20:56:09.216507 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] deleting parent snapshot volume-bae2c97c-a30a-424b-bc18-c9977d768266.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 20:56:09.761663 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] Acquiring lock "cinder-f83c038c-0838-41d4-b421-20d53a0dec11-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 20:56:09.775619 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] Lock "cinder-f83c038c-0838-41d4-b421-20d53a0dec11-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 20:56:09.828581 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] Created reservations ['0b177a45-6313-4832-8688-11318a6e447d', '17113731-8a3d-444d-926b-2398e3e5cdb1', '0f6ea6ac-046f-45c8-a385-c97e85f35fd6', '44ddb15c-23f8-4287-a0c0-e04ba8f51c14'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 20:56:09.840946 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:56:09.881337 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 20:56:09.881791 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] Volume volume-f83c038c-0838-41d4-b421-20d53a0dec11 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 20:56:09.886306 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] deleting rbd volume volume-f83c038c-0838-41d4-b421-20d53a0dec11 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 20:56:09.937321 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:56:09.984478 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:56:10.036520 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:56:10.043261 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] Deleted volume successfully. Oct 01 20:56:10.064817 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-3e7afda6-c071-4ef2-b8f2-624955d1ea62 req-f602b189-6376-4585-81c0-1813d5f4d62d tempest-LiveMigrationTest-1978574502 None] Lock "cinder-bae2c97c-a30a-424b-bc18-c9977d768266-delete_volume" released by "delete_volume" :: held 1.434s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 20:56:10.174140 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] Created reservations ['75950dc5-95e9-4142-9333-62c103e71a88', 'f5a9ba23-7263-4b77-85c5-fd659db83700', '1f047f2b-a697-426a-a11d-621bdfc1e69b', 'b34ed75f-82eb-46c4-8673-205719b0f820'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 20:56:10.253905 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:56:10.288598 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:56:10.331991 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:56:10.336277 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] Deleted volume successfully. Oct 01 20:56:10.350105 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-75f7aa83-a46a-4678-afde-f41e68ab965a tempest-LiveMigrationTest-1978574502 None] Lock "cinder-f83c038c-0838-41d4-b421-20d53a0dec11-delete_volume" released by "delete_volume" :: held 0.574s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 20:56:22.863821 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=35,cinder:INSERT=5,cinder:UPDATE=15 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:23.704079 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:56:23.705146 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:56:23.793805 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:56:23.895553 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:56:32.864204 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=30,cinder:UPDATE=20,cinder:INSERT=3,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:56:52.864824 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:12.868887 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:21.355511 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-66a428aa-0cab-4f19-a7d0-9df77c9a5f15 req-a686dadf-d2f2-4aa4-8ab5-8336cfa98fbd tempest-TaggedBootDevicesTest_v242-711726808 None] Terminate volume connection completed successfully. Oct 01 20:57:21.356099 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-66a428aa-0cab-4f19-a7d0-9df77c9a5f15 req-a686dadf-d2f2-4aa4-8ab5-8336cfa98fbd tempest-TaggedBootDevicesTest_v242-711726808 None] Deleting attachment 54b22890-654f-40e9-95ac-8e274986595a. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 20:57:21.601194 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-66a428aa-0cab-4f19-a7d0-9df77c9a5f15 req-1a2b25d8-5cb7-40c2-b28e-097be1e95e66 tempest-TaggedBootDevicesTest_v242-711726808 None] Terminate volume connection completed successfully. Oct 01 20:57:21.601616 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-66a428aa-0cab-4f19-a7d0-9df77c9a5f15 req-1a2b25d8-5cb7-40c2-b28e-097be1e95e66 tempest-TaggedBootDevicesTest_v242-711726808 None] Deleting attachment 26aacba7-4bbb-468a-9e74-7cb9fa616eb1. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 20:57:21.876558 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-66a428aa-0cab-4f19-a7d0-9df77c9a5f15 req-46c5e446-727f-44d2-a2ac-cb5490ed2677 tempest-TaggedBootDevicesTest_v242-711726808 None] Terminate volume connection completed successfully. Oct 01 20:57:21.876984 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-66a428aa-0cab-4f19-a7d0-9df77c9a5f15 req-46c5e446-727f-44d2-a2ac-cb5490ed2677 tempest-TaggedBootDevicesTest_v242-711726808 None] Deleting attachment d50a1401-4d34-4241-a195-ce773ee990c1. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 20:57:23.704200 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:57:23.705789 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:57:23.786829 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:57:23.838371 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:57:31.871000 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:32.881371 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=10,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:57:34.078052 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] Acquiring lock "cinder-d4764752-297f-4f5f-bd4a-009489d9151b-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 20:57:34.108339 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] Lock "cinder-d4764752-297f-4f5f-bd4a-009489d9151b-delete_volume" acquired by "delete_volume" :: waited 0.030s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 20:57:34.210048 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:57:34.368009 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 20:57:34.368346 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume volume-d4764752-297f-4f5f-bd4a-009489d9151b is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 20:57:34.379517 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] deleting rbd volume volume-d4764752-297f-4f5f-bd4a-009489d9151b {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 20:57:34.758033 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] Created reservations ['b618dc6e-236a-4d4a-8571-50b9614fad16', '9adfe061-d96c-459e-93ad-6ee3fa23819e', 'f3eda3c1-4b92-4578-9ae1-8847e4d3eb56', 'd905a1ae-a4ca-4032-926b-23d3685671a8'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 20:57:34.868697 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:57:34.929331 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:57:34.971075 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:57:34.976199 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] Deleted volume successfully. Oct 01 20:57:34.992292 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-154a287f-9907-4a1e-8526-51a268015390 tempest-TaggedBootDevicesTest_v242-711726808 None] Lock "cinder-d4764752-297f-4f5f-bd4a-009489d9151b-delete_volume" released by "delete_volume" :: held 0.884s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 20:57:35.361147 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] Acquiring lock "cinder-e3be1f38-f8d0-4c82-a901-0c7a1abac289-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 20:57:35.380222 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] Lock "cinder-e3be1f38-f8d0-4c82-a901-0c7a1abac289-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 20:57:35.471571 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:57:35.547636 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 20:57:35.548246 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume volume-e3be1f38-f8d0-4c82-a901-0c7a1abac289 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 20:57:35.555126 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] deleting rbd volume volume-e3be1f38-f8d0-4c82-a901-0c7a1abac289 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 20:57:35.916864 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] Created reservations ['5e26f202-38e2-495f-8bf4-462b6f6ad4ff', 'bb1c407d-bacf-4641-819f-ba3a434b7b3f', 'a5b7bd4c-c7d4-440e-9005-681142c77772', '681f52c7-9a20-4248-8e1f-ed199f26f9fa'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 20:57:36.006443 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:57:36.067241 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:57:36.130265 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:57:36.136727 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] Deleted volume successfully. Oct 01 20:57:36.155699 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7284ebf9-eb3f-43dd-a623-4f65601afe80 tempest-TaggedBootDevicesTest_v242-711726808 None] Lock "cinder-e3be1f38-f8d0-4c82-a901-0c7a1abac289-delete_volume" released by "delete_volume" :: held 0.776s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 20:57:36.537430 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] Acquiring lock "cinder-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 20:57:36.551432 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] Lock "cinder-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 20:57:36.599937 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:57:36.686363 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 20:57:36.686831 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] volume has clone snapshot(s) {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1378}} Oct 01 20:57:36.687945 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 20:57:36.839763 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] Created reservations ['d6b839ac-69ee-4b4c-988a-ca57d407b8eb', 'fd60d8ee-aad3-4a19-8791-fa7fe51b10e4', '811291cd-3831-49c5-a8a0-6c112aebe44a', '416811f4-f0a8-42fa-ada0-0d95a62ea7c2'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 20:57:36.959825 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:57:37.059797 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:57:37.118776 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:57:37.123954 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] Deleted volume successfully. Oct 01 20:57:37.146703 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1146ab28-d559-43eb-9512-580a105f1721 tempest-TaggedBootDevicesTest_v242-711726808 None] Lock "cinder-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8-delete_volume" released by "delete_volume" :: held 0.597s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 20:57:52.886098 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=41,cinder:UPDATE=19,cinder:INSERT=3,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:12.883977 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=42,cinder:INSERT=9,cinder:UPDATE=32,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:23.702942 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:58:23.704224 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:58:23.856173 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:58:23.941945 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:58:32.879012 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:58:52.883846 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:02.890386 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:12.895868 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:22.889935 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:23.708327 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 20:59:23.709040 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:59:23.786016 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 20:59:23.870202 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 20:59:52.903499 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 20:59:52.945882 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:11.932531 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:00:11.988124 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Flow 'volume_create_manager' (24189d97-8917-4af7-9440-6ab872614941) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:00:12.006521 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8fe5da52-76b1-4f6d-a31d-034a7402aceb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:12.052232 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8fe5da52-76b1-4f6d-a31d-034a7402aceb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:00:11Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachSCSIVolumeTestJSON-volume-1069781222',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=08131bb6-39bf-46fc-b169-736844ac1286,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='bcdb77df347145639f2de60eb820a3ca',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:00:12Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:00:12Z,use_quota=True,user_id='ccbf2c814364466baca8dfcabd2c13f9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:12.062272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b349bef5-433c-4a3c-974a-f0157b4929ef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:12.067816 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b349bef5-433c-4a3c-974a-f0157b4929ef) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:12.073383 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e11482b4-24d6-4322-818f-d48bf2edb084) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:12.081173 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e11482b4-24d6-4322-818f-d48bf2edb084) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '08131bb6-39bf-46fc-b169-736844ac1286', 'volume_name': 'volume-08131bb6-39bf-46fc-b169-736844ac1286', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:12.088671 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f8ac79d2-9225-4e8c-8d25-08d4cf52f60f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:12.093175 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f8ac79d2-9225-4e8c-8d25-08d4cf52f60f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:12.119075 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (450a9e76-8654-4f0a-9119-e5f1daa3c18d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:12.121050 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Volume 08131bb6-39bf-46fc-b169-736844ac1286: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-08131bb6-39bf-46fc-b169-736844ac1286', 'volume_size': 1} Oct 01 21:00:12.124024 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] creating volume 'volume-08131bb6-39bf-46fc-b169-736844ac1286' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:00:12.125730 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:00:12.284107 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:00:12.284467 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:00:12.288606 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (450a9e76-8654-4f0a-9119-e5f1daa3c18d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-08131bb6-39bf-46fc-b169-736844ac1286', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:12.295603 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d0626690-9ea3-49be-8bb3-68f7db6acb19) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:12.318222 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Volume volume-08131bb6-39bf-46fc-b169-736844ac1286 (08131bb6-39bf-46fc-b169-736844ac1286): created successfully Oct 01 21:00:12.323930 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d0626690-9ea3-49be-8bb3-68f7db6acb19) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:12.335183 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Flow 'volume_create_manager' (24189d97-8917-4af7-9440-6ab872614941) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:00:12.370150 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7848cf5e-3e6d-4734-8b87-1828a2fae6db tempest-AttachSCSIVolumeTestJSON-1704747336 None] Created volume successfully. Oct 01 21:00:13.815080 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-6142fe81-86a0-409a-83c8-67dce0222e36 req-00e38cfe-974e-473c-b338-d2992b0d2b15 tempest-AttachSCSIVolumeTestJSON-1704747336 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:00:15.577321 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-6142fe81-86a0-409a-83c8-67dce0222e36 req-00e38cfe-974e-473c-b338-d2992b0d2b15 tempest-AttachSCSIVolumeTestJSON-1704747336 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.756s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:00:15.579926 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-6142fe81-86a0-409a-83c8-67dce0222e36 req-00e38cfe-974e-473c-b338-d2992b0d2b15 tempest-AttachSCSIVolumeTestJSON-1704747336 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-08131bb6-39bf-46fc-b169-736844ac1286', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '08131bb6-39bf-46fc-b169-736844ac1286', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:00:15.635275 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-6142fe81-86a0-409a-83c8-67dce0222e36 req-00e38cfe-974e-473c-b338-d2992b0d2b15 tempest-AttachSCSIVolumeTestJSON-1704747336 None] Connection info returned from driver {'name': 'volumes/volume-08131bb6-39bf-46fc-b169-736844ac1286', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '08131bb6-39bf-46fc-b169-736844ac1286', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '93a7df5f-1ffb-46d5-aa00-dbe648a3a8fb'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:00:15.704330 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-6142fe81-86a0-409a-83c8-67dce0222e36 req-00e38cfe-974e-473c-b338-d2992b0d2b15 tempest-AttachSCSIVolumeTestJSON-1704747336 None] attachment_update completed successfully. Oct 01 21:00:22.488126 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-8023f697-ee7c-44ac-8c58-185e5b329aec req-d64eecc6-7e05-4e6a-bc41-0f261e5ba695 tempest-AttachSCSIVolumeTestJSON-1704747336 None] Terminate volume connection completed successfully. Oct 01 21:00:22.488943 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-8023f697-ee7c-44ac-8c58-185e5b329aec req-d64eecc6-7e05-4e6a-bc41-0f261e5ba695 tempest-AttachSCSIVolumeTestJSON-1704747336 None] Deleting attachment 93a7df5f-1ffb-46d5-aa00-dbe648a3a8fb. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:00:23.711260 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:00:23.712950 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:00:23.803463 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:00:23.893000 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:00:31.765654 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=4,cinder:SELECT=24,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:31.768588 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=20,cinder:UPDATE=6 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:00:31.804504 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:00:31.835793 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Flow 'volume_create_manager' (9e6f3612-edbe-4bef-acad-102cf416f62c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:00:31.849764 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (682aed64-48c9-4c85-b60c-a32810cd06f3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:31.880148 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (682aed64-48c9-4c85-b60c-a32810cd06f3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:00:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-1211994292',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8854826b-5d7e-43c4-9404-d505ad354cdc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d27e9cda0a6d47d4acdbadc890b996c9',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:00:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:00:32Z,use_quota=True,user_id='69002d46d4e54eadadf911ec3589d95c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:31.887907 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d15faa7c-25cd-4cee-83e6-018fd7befa17) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:31.899474 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d15faa7c-25cd-4cee-83e6-018fd7befa17) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:31.906621 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (103d7d10-4b1d-4235-bac2-5ae28db8d4b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:31.929887 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (103d7d10-4b1d-4235-bac2-5ae28db8d4b1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8854826b-5d7e-43c4-9404-d505ad354cdc', 'volume_name': 'volume-8854826b-5d7e-43c4-9404-d505ad354cdc', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:31.936822 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8b329823-b1f3-4a4d-aa3c-2a6c60cd9e5a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:31.942928 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8b329823-b1f3-4a4d-aa3c-2a6c60cd9e5a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:31.948027 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (53758d63-cba0-477c-a39e-6c1d53e9c7d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:31.949474 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Volume 8854826b-5d7e-43c4-9404-d505ad354cdc: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8854826b-5d7e-43c4-9404-d505ad354cdc', 'volume_size': 1} Oct 01 21:00:31.950152 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] creating volume 'volume-8854826b-5d7e-43c4-9404-d505ad354cdc' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:00:31.951023 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:00:32.066080 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:00:32.066877 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:00:32.072677 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (53758d63-cba0-477c-a39e-6c1d53e9c7d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8854826b-5d7e-43c4-9404-d505ad354cdc', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:32.078483 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (13c569d7-e81a-4e4d-b860-3ab7fe19e080) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:00:32.092167 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Volume volume-8854826b-5d7e-43c4-9404-d505ad354cdc (8854826b-5d7e-43c4-9404-d505ad354cdc): created successfully Oct 01 21:00:32.096987 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (13c569d7-e81a-4e4d-b860-3ab7fe19e080) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:00:32.104767 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Flow 'volume_create_manager' (9e6f3612-edbe-4bef-acad-102cf416f62c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:00:32.123275 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8e392f93-e204-460b-ad24-ea7166116fd6 tempest-VolumesAdminNegativeTest-1221345461 None] Created volume successfully. Oct 01 21:00:34.048682 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-36b5eba3-26d8-4a09-9c91-cdc48799157c req-75828d83-121e-4755-a4e4-e890c7030159 tempest-VolumesAdminNegativeTest-1221345461 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:00:35.137824 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-36b5eba3-26d8-4a09-9c91-cdc48799157c req-75828d83-121e-4755-a4e4-e890c7030159 tempest-VolumesAdminNegativeTest-1221345461 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.088s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:00:35.139375 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-36b5eba3-26d8-4a09-9c91-cdc48799157c req-75828d83-121e-4755-a4e4-e890c7030159 tempest-VolumesAdminNegativeTest-1221345461 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-8854826b-5d7e-43c4-9404-d505ad354cdc', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '8854826b-5d7e-43c4-9404-d505ad354cdc', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:00:35.166464 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-36b5eba3-26d8-4a09-9c91-cdc48799157c req-75828d83-121e-4755-a4e4-e890c7030159 tempest-VolumesAdminNegativeTest-1221345461 None] Connection info returned from driver {'name': 'volumes/volume-8854826b-5d7e-43c4-9404-d505ad354cdc', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '8854826b-5d7e-43c4-9404-d505ad354cdc', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'ed79fb3c-a19b-4d10-923d-cc5c0b197e37'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:00:35.244188 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-36b5eba3-26d8-4a09-9c91-cdc48799157c req-75828d83-121e-4755-a4e4-e890c7030159 tempest-VolumesAdminNegativeTest-1221345461 None] attachment_update completed successfully. Oct 01 21:00:42.724216 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-9ad0ace0-8eef-4b3d-80fe-2132fd385637 req-13dfe8e2-20dd-4310-9c7c-0e5fcc68a16c tempest-VolumesAdminNegativeTest-1221345461 None] Terminate volume connection completed successfully. Oct 01 21:00:42.724830 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-9ad0ace0-8eef-4b3d-80fe-2132fd385637 req-13dfe8e2-20dd-4310-9c7c-0e5fcc68a16c tempest-VolumesAdminNegativeTest-1221345461 None] Deleting attachment ed79fb3c-a19b-4d10-923d-cc5c0b197e37. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:00:43.524751 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] Acquiring lock "cinder-08131bb6-39bf-46fc-b169-736844ac1286-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:00:43.556246 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] Lock "cinder-08131bb6-39bf-46fc-b169-736844ac1286-delete_volume" acquired by "delete_volume" :: waited 0.031s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:00:43.614315 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:00:43.700021 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:00:43.700610 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] Volume volume-08131bb6-39bf-46fc-b169-736844ac1286 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:00:43.711681 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] deleting rbd volume volume-08131bb6-39bf-46fc-b169-736844ac1286 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:00:44.064967 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] Created reservations ['7039029b-30c1-488f-a2e5-219113ef80b3', 'e9f9ba8f-6e90-4b43-88a6-6cabaaebeb4f', 'f705dc38-ef4d-4846-b974-52d0971787d3', '0fc0e82e-5e8f-47fc-9500-62a0b6a73ecb'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:00:44.164243 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:00:44.230812 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:00:44.294200 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:00:44.299291 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] Deleted volume successfully. Oct 01 21:00:44.313789 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-91681a2e-f549-47a3-83e7-42edc5e33bb6 tempest-AttachSCSIVolumeTestJSON-1704747336 None] Lock "cinder-08131bb6-39bf-46fc-b169-736844ac1286-delete_volume" released by "delete_volume" :: held 0.757s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:00:54.303204 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=30,cinder:UPDATE=11,cinder:INSERT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:02.905260 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=35,cinder:DELETE=2,cinder:UPDATE=14 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:03.461664 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:01:03.485012 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Flow 'volume_create_manager' (ca19a444-0106-4c88-abec-04ccf91f6d7c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:01:03.492730 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0c7d161d-cd18-48d8-9837-dfe404d9ed7b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:01:03.524318 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0c7d161d-cd18-48d8-9837-dfe404d9ed7b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:01:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-877443900',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d27e9cda0a6d47d4acdbadc890b996c9',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:01:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:01:03Z,use_quota=True,user_id='69002d46d4e54eadadf911ec3589d95c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:01:03.531763 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (629dd013-0c44-44c1-b080-25a75522f233) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:01:03.542397 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (629dd013-0c44-44c1-b080-25a75522f233) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:01:03.548801 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6850a153-a055-4aee-98ef-11afa2a198a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:01:03.554990 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6850a153-a055-4aee-98ef-11afa2a198a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8', 'volume_name': 'volume-a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:01:03.575850 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (42a8847b-5d3b-419a-a47b-3d62cd48045f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:01:03.581732 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (42a8847b-5d3b-419a-a47b-3d62cd48045f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:01:03.586481 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (05cc66b2-189b-4ccb-af52-926b1d3c7d52) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:01:03.587715 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Volume a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8', 'volume_size': 1} Oct 01 21:01:03.589331 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] creating volume 'volume-a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:01:03.590321 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:01:03.716628 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:01:03.717208 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:01:03.721003 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (05cc66b2-189b-4ccb-af52-926b1d3c7d52) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:01:03.735192 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d9aaa3f8-17b0-45e0-b535-48a5edfda8a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:01:03.755866 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Volume volume-a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8 (a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8): created successfully Oct 01 21:01:03.759951 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d9aaa3f8-17b0-45e0-b535-48a5edfda8a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:01:03.767751 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Flow 'volume_create_manager' (ca19a444-0106-4c88-abec-04ccf91f6d7c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:01:03.780294 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d89cccf7-ac84-4dfc-85ed-cbdad39f2fd9 tempest-VolumesAdminNegativeTest-1221345461 None] Created volume successfully. Oct 01 21:01:04.673601 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] Acquiring lock "cinder-a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:01:04.695626 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] Lock "cinder-a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:01:04.767253 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:01:04.847857 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:01:04.848289 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] Volume volume-a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:01:04.861198 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] deleting rbd volume volume-a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:01:05.275740 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] Created reservations ['b0263ac9-ad8d-44c8-871e-17e0d7399a7d', 'a65403bb-795b-4a52-9091-a3f4e65a58a7', '2b5d26a3-1ce2-44b2-905b-e8fa3909529e', 'bc51efe7-1850-4dad-9e64-5c60f6b349ea'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:01:05.339023 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:01:05.411533 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:01:05.459572 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:01:05.465729 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] Deleted volume successfully. Oct 01 21:01:05.480918 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3fcb3756-77ad-4676-b4ae-115c3df07193 tempest-VolumesAdminNegativeTest-1221345461 None] Lock "cinder-a7eac5bb-18dd-4a33-831a-6a51ec4fd4c8-delete_volume" released by "delete_volume" :: held 0.788s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:01:11.678578 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] Acquiring lock "cinder-8854826b-5d7e-43c4-9404-d505ad354cdc-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:01:11.713702 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] Lock "cinder-8854826b-5d7e-43c4-9404-d505ad354cdc-delete_volume" acquired by "delete_volume" :: waited 0.035s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:01:11.780054 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:01:11.883051 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:01:11.883488 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] Volume volume-8854826b-5d7e-43c4-9404-d505ad354cdc is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:01:11.888983 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] deleting rbd volume volume-8854826b-5d7e-43c4-9404-d505ad354cdc {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:01:12.287783 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] Created reservations ['529fd6c1-1c2d-4ede-a1bb-96879abd3bfa', '55c7a155-b497-4b54-9df7-aef384fef3b3', '42ef2296-b614-4426-be99-56e1a95414a4', '3293e38c-fa0a-4540-bd34-4d503f74036d'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:01:12.377615 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:01:12.450461 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:01:12.512707 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:01:12.517197 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] Deleted volume successfully. Oct 01 21:01:12.536375 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e70f65fe-2105-49c8-aa90-a1158d1fb4b7 tempest-VolumesAdminNegativeTest-1221345461 None] Lock "cinder-8854826b-5d7e-43c4-9404-d505ad354cdc-delete_volume" released by "delete_volume" :: held 0.822s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:01:22.923526 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=23,cinder:UPDATE=23,cinder:DELETE=2,cinder:INSERT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:22.926122 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=38,cinder:UPDATE=14,cinder:INSERT=4,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:01:23.709289 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:01:23.709959 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:01:23.782441 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:01:23.839134 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:01:42.906595 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:02.906253 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=10,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:22.907963 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:23.706307 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:02:23.707331 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:02:23.809166 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:02:23.901428 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:02:42.926058 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:02:43.616907 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:02:43.639199 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Flow 'volume_create_manager' (5b97b12f-f6c2-4c2b-a174-4f4b693dfc81) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:02:43.651216 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1b1b0fd6-7785-4499-acc3-d515e5a9a76a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:02:43.673025 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1b1b0fd6-7785-4499-acc3-d515e5a9a76a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:02:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerRescueNegativeTestJSON-volume-1467105834',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d75ab76e-9835-456b-8a91-4e4c433fd49b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='10fefee3d6a6430682316cd94b859f1e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:02:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:02:44Z,use_quota=True,user_id='6bda28994cb14f92b014a147a7e74476',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:02:43.680666 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fc5d5dd6-4cce-4cbd-ba9f-391a42d2917e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:02:43.695740 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fc5d5dd6-4cce-4cbd-ba9f-391a42d2917e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:02:43.704157 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (07ef6960-50ba-4a2a-bee3-7d8846287db3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:02:43.712232 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (07ef6960-50ba-4a2a-bee3-7d8846287db3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd75ab76e-9835-456b-8a91-4e4c433fd49b', 'volume_name': 'volume-d75ab76e-9835-456b-8a91-4e4c433fd49b', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:02:43.732397 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ebc688ea-5759-4b38-af25-a48c7977e8ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:02:43.745334 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ebc688ea-5759-4b38-af25-a48c7977e8ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:02:43.780314 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1416a79e-a96e-4c5f-92ea-8bdda4bd8693) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:02:43.784291 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Volume d75ab76e-9835-456b-8a91-4e4c433fd49b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d75ab76e-9835-456b-8a91-4e4c433fd49b', 'volume_size': 1} Oct 01 21:02:43.785829 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] creating volume 'volume-d75ab76e-9835-456b-8a91-4e4c433fd49b' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:02:43.791425 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:02:43.951336 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:02:43.951914 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:02:43.964538 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1416a79e-a96e-4c5f-92ea-8bdda4bd8693) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d75ab76e-9835-456b-8a91-4e4c433fd49b', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:02:43.985301 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5df2f065-1428-4d4d-8d4f-9529e79062e4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:02:44.012643 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Volume volume-d75ab76e-9835-456b-8a91-4e4c433fd49b (d75ab76e-9835-456b-8a91-4e4c433fd49b): created successfully Oct 01 21:02:44.016204 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5df2f065-1428-4d4d-8d4f-9529e79062e4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:02:44.065241 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Flow 'volume_create_manager' (5b97b12f-f6c2-4c2b-a174-4f4b693dfc81) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:02:44.085859 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0f5266c4-b009-461f-885a-6bb7b44be299 tempest-ServerRescueNegativeTestJSON-894078329 None] Created volume successfully. Oct 01 21:03:02.919822 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=10,cinder:UPDATE=4,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:22.921827 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=11,cinder:UPDATE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:23.706675 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:03:23.708205 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:03:23.846555 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:03:23.949979 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:03:34.877654 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:03:34.899886 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:03:34.937828 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Flow 'volume_create_manager' (84b76d82-c60f-4309-bd5e-3c78bdef9d81) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:03:34.945422 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a0639a75-1946-4c9c-b6f7-31be4ab871f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:03:34.968692 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a0639a75-1946-4c9c-b6f7-31be4ab871f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:03:35Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerRescueNegativeTestJSON-volume-1762245710',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1067c786-686b-41aa-b9ca-31ea24d10308,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='10fefee3d6a6430682316cd94b859f1e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:03:35Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:03:35Z,use_quota=True,user_id='6bda28994cb14f92b014a147a7e74476',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:03:34.975671 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (46ebc5a0-4eb7-4433-87d8-1876a327deee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:03:34.982071 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (46ebc5a0-4eb7-4433-87d8-1876a327deee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:03:34.987751 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (71ddd84b-28a2-4e52-9828-5d9bb0d95216) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:03:34.993472 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (71ddd84b-28a2-4e52-9828-5d9bb0d95216) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1067c786-686b-41aa-b9ca-31ea24d10308', 'volume_name': 'volume-1067c786-686b-41aa-b9ca-31ea24d10308', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:03:34.999425 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (81a93284-7084-42d9-9595-dbadd247688d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:03:35.010250 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (81a93284-7084-42d9-9595-dbadd247688d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:03:35.015432 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1580835c-49f2-4944-94b3-8a5b75b343ac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:03:35.016567 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Volume 1067c786-686b-41aa-b9ca-31ea24d10308: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1067c786-686b-41aa-b9ca-31ea24d10308', 'volume_size': 1} Oct 01 21:03:35.017084 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] creating volume 'volume-1067c786-686b-41aa-b9ca-31ea24d10308' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:03:35.017982 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:03:35.115641 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:03:35.116011 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:03:35.120151 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1580835c-49f2-4944-94b3-8a5b75b343ac) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1067c786-686b-41aa-b9ca-31ea24d10308', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:03:35.124852 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3739e74c-f3ed-49c6-934a-1165e13aeb71) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:03:35.157873 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Volume volume-1067c786-686b-41aa-b9ca-31ea24d10308 (1067c786-686b-41aa-b9ca-31ea24d10308): created successfully Oct 01 21:03:35.162248 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3739e74c-f3ed-49c6-934a-1165e13aeb71) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:03:35.174846 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Flow 'volume_create_manager' (84b76d82-c60f-4309-bd5e-3c78bdef9d81) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:03:35.196002 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e29f34e9-04b3-4a64-995b-f81ee1bc0891 tempest-ServerRescueNegativeTestJSON-894078329 None] Created volume successfully. Oct 01 21:03:52.930179 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:02.928022 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=10,cinder:UPDATE=4,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:12.943218 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:17.645657 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-13f8b89c-0258-4c3d-9a1c-3c3ef7f08b0a tempest-GroupsV320Test-861866939 None] Group tempest-GroupsV320Test-Group-1613319013: creating Oct 01 21:04:17.698446 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-13f8b89c-0258-4c3d-9a1c-3c3ef7f08b0a tempest-GroupsV320Test-861866939 None] Group tempest-GroupsV320Test-Group-1613319013: created successfully Oct 01 21:04:17.713793 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-13f8b89c-0258-4c3d-9a1c-3c3ef7f08b0a tempest-GroupsV320Test-861866939 None] Create group completed successfully. Oct 01 21:04:19.531826 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-6b7969ea-648d-405d-9211-33f3ab0568c2 tempest-GroupsV320Test-861866939 None] Created reservations ['1b49fe46-d117-4114-9c63-ffe15938b718'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:04:19.612112 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6b7969ea-648d-405d-9211-33f3ab0568c2 tempest-GroupsV320Test-861866939 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:19.690438 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6b7969ea-648d-405d-9211-33f3ab0568c2 tempest-GroupsV320Test-861866939 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:19.790699 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-6b7969ea-648d-405d-9211-33f3ab0568c2 tempest-GroupsV320Test-861866939 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:04:19.797233 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6b7969ea-648d-405d-9211-33f3ab0568c2 tempest-GroupsV320Test-861866939 None] Delete group completed successfully. Oct 01 21:04:23.706238 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:04:23.706909 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:23.782587 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:23.855399 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:04:27.291054 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:04:27.313467 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Flow 'volume_create_manager' (9ebc425b-0994-494e-a867-a98d8c192885) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:04:27.320757 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c6c1b538-819f-4c6b-9be6-06329afb8535) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:27.342634 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c6c1b538-819f-4c6b-9be6-06329afb8535) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:04:27Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeSnapshotQuotasNegativeTestJSON-Volume-2049145168',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1388f946-d060-4937-8193-bd16c9c5b05b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='27cbdc934b2945f3b1765acee67fc104',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:04:27Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:04:27Z,use_quota=True,user_id='6e66430017414257a4626b418ee714eb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:27.367975 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (86793833-17c0-431d-ba25-29d2d27239ce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:27.378921 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (86793833-17c0-431d-ba25-29d2d27239ce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:27.386047 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (30ee542d-3240-49b9-a865-aacc6c6f206c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:27.391522 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (30ee542d-3240-49b9-a865-aacc6c6f206c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1388f946-d060-4937-8193-bd16c9c5b05b', 'volume_name': 'volume-1388f946-d060-4937-8193-bd16c9c5b05b', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:27.397203 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0d59a7b8-e79b-4b0a-89cb-ff01f5c6cb59) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:27.405074 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0d59a7b8-e79b-4b0a-89cb-ff01f5c6cb59) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:27.411268 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2ebb7b4d-df93-4923-899e-221a9ede91ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:27.416560 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Volume 1388f946-d060-4937-8193-bd16c9c5b05b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1388f946-d060-4937-8193-bd16c9c5b05b', 'volume_size': 1} Oct 01 21:04:27.417200 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] creating volume 'volume-1388f946-d060-4937-8193-bd16c9c5b05b' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:04:27.418071 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:27.521801 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:04:27.522183 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:04:27.525906 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2ebb7b4d-df93-4923-899e-221a9ede91ca) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1388f946-d060-4937-8193-bd16c9c5b05b', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:27.530718 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f8d19fff-e00c-4770-a47a-062fe8f31c8c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:27.557646 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Volume volume-1388f946-d060-4937-8193-bd16c9c5b05b (1388f946-d060-4937-8193-bd16c9c5b05b): created successfully Oct 01 21:04:27.561941 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f8d19fff-e00c-4770-a47a-062fe8f31c8c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:27.570826 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Flow 'volume_create_manager' (9ebc425b-0994-494e-a867-a98d8c192885) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:04:27.587119 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3f18c88e-db59-4900-85ba-43f2decd354d tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Created volume successfully. Oct 01 21:04:28.733789 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ece4a3f7-2e12-4347-87ba-51029d614c62 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:29.292132 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ece4a3f7-2e12-4347-87ba-51029d614c62 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Create snapshot completed successfully Oct 01 21:04:30.325039 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1589a1eb-af56-4d39-96e8-bc0999a706bb tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Acquiring lock "cinder-479edace-8ca4-474c-ae18-5ce680e47c5e-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:04:30.334891 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1589a1eb-af56-4d39-96e8-bc0999a706bb tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Lock "cinder-479edace-8ca4-474c-ae18-5ce680e47c5e-delete_snapshot" acquired by "delete_snapshot" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:04:30.337397 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1589a1eb-af56-4d39-96e8-bc0999a706bb tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:31.328028 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-1589a1eb-af56-4d39-96e8-bc0999a706bb tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Deprecated: Default quota for resource: snapshots_ceph is set by the default quota flag: quota_snapshots_ceph, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:04:31.349366 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-1589a1eb-af56-4d39-96e8-bc0999a706bb tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Created reservations ['60bbebd0-82c8-450f-91f7-ba2b05100eaf', 'b9b723b3-38b2-4701-989f-8d20cc1a5e9b', '1fb5f90e-3e46-44a9-8b47-2575b098b26c', '3353be77-a978-48cf-9899-7f9feabc7cc5'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:04:31.446377 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1589a1eb-af56-4d39-96e8-bc0999a706bb tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Delete snapshot completed successfully. Oct 01 21:04:31.461148 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1589a1eb-af56-4d39-96e8-bc0999a706bb tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Lock "cinder-479edace-8ca4-474c-ae18-5ce680e47c5e-delete_snapshot" released by "delete_snapshot" :: held 1.126s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:04:31.524253 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Acquiring lock "cinder-1388f946-d060-4937-8193-bd16c9c5b05b-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:04:31.539096 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Lock "cinder-1388f946-d060-4937-8193-bd16c9c5b05b-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:04:31.601778 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:31.709832 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:04:31.710636 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Volume volume-1388f946-d060-4937-8193-bd16c9c5b05b is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:04:31.719854 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] deleting rbd volume volume-1388f946-d060-4937-8193-bd16c9c5b05b {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:04:32.015975 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Created reservations ['98f5b10f-c0ad-40ff-ab15-65d05fef9a9b', '7b8a675f-018e-4b7d-933e-fba9260879c3', 'ce3ce4c5-9439-4146-a04a-521982bf1a1f', '5689811a-3efb-4626-9484-8e3e1cb49d50'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:04:32.066662 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:32.123403 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:32.187567 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:04:32.193021 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Deleted volume successfully. Oct 01 21:04:32.215363 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-8443ff63-abfa-4ad5-99b6-afecfbcef0f5 tempest-VolumeSnapshotQuotasNegativeTestJSON-576098729 None] Lock "cinder-1388f946-d060-4937-8193-bd16c9c5b05b-delete_volume" released by "delete_volume" :: held 0.676s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:04:34.603889 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=48,cinder:UPDATE=22,cinder:DELETE=2,cinder:INSERT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:34.636734 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:04:34.684013 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Flow 'volume_create_manager' (10458ad4-b0ab-4dbf-b06a-eb197f6f9f24) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:04:34.699240 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b35a1675-3bfa-46e3-8158-f881137edeb5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:34.762015 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b35a1675-3bfa-46e3-8158-f881137edeb5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:04:34Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotsTestJSON-volume-561882731',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3155b986-3afe-4c50-83df-264e88b679e1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e8be683460bb4157824a7f999d0dd106',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:04:35Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:04:35Z,use_quota=True,user_id='644bb91fc7d14468a59c399eb3df8473',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:34.769713 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6dbd9533-c55f-4583-b906-bb140042e9db) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:34.775943 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6dbd9533-c55f-4583-b906-bb140042e9db) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:34.782257 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (872b34e6-a47f-4e0d-b11a-b31f2174d590) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:34.787970 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (872b34e6-a47f-4e0d-b11a-b31f2174d590) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '3155b986-3afe-4c50-83df-264e88b679e1', 'volume_name': 'volume-3155b986-3afe-4c50-83df-264e88b679e1', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:34.794683 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (382894a5-7797-46e7-a245-7bffa201fb62) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:34.800604 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (382894a5-7797-46e7-a245-7bffa201fb62) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:34.807898 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (107d042a-948d-4f19-be7d-8887f72be816) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:34.809842 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Volume 3155b986-3afe-4c50-83df-264e88b679e1: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-3155b986-3afe-4c50-83df-264e88b679e1', 'volume_size': 1} Oct 01 21:04:34.810627 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] creating volume 'volume-3155b986-3afe-4c50-83df-264e88b679e1' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:04:34.811830 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:34.942051 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:04:34.942461 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:04:34.946938 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (107d042a-948d-4f19-be7d-8887f72be816) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3155b986-3afe-4c50-83df-264e88b679e1', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:34.952628 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a1726cdb-8756-41d8-b1f8-c9e9335b0ec4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:34.964827 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Volume volume-3155b986-3afe-4c50-83df-264e88b679e1 (3155b986-3afe-4c50-83df-264e88b679e1): created successfully Oct 01 21:04:34.969548 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a1726cdb-8756-41d8-b1f8-c9e9335b0ec4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:34.975261 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Flow 'volume_create_manager' (10458ad4-b0ab-4dbf-b06a-eb197f6f9f24) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:04:34.987883 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-abb807b1-61cd-490b-b875-52eaae9fb491 req-2199b8b5-25d9-4a10-a2fd-2cb74e546628 tempest-VolumesSnapshotsTestJSON-740423316 None] Created volume successfully. Oct 01 21:04:36.492587 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-19a08881-6ac7-4847-980a-ab2e5be5acbd req-cef28dd2-44b5-4339-8052-815a8d3aedbe tempest-VolumesSnapshotsTestJSON-740423316 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:37.285228 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-19a08881-6ac7-4847-980a-ab2e5be5acbd req-cef28dd2-44b5-4339-8052-815a8d3aedbe tempest-VolumesSnapshotsTestJSON-740423316 None] Create snapshot completed successfully Oct 01 21:04:38.869923 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:04:38.915104 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Flow 'volume_create_manager' (e56d7a6b-94b4-4797-81ea-57f1d2bb0521) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:04:38.937049 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (016453b4-fa33-4624-bb28-17ad972dcc1f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:38.978448 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (016453b4-fa33-4624-bb28-17ad972dcc1f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:04:38Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Volume-596301630',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=356b6fbc-fccb-4b94-95cc-e6051cb7e306,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='cd3526ab00f3485e98db7f02e57bea4d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:04:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:04:39Z,use_quota=True,user_id='477eb33574da473284d09e765825bcc6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:38.990746 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5eda8a99-a746-4ab5-be27-fc0ef5bb45c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:38.995653 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5eda8a99-a746-4ab5-be27-fc0ef5bb45c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:39.017426 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1c253851-0cfe-4cd7-b499-2260d9da4992) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:39.025964 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1c253851-0cfe-4cd7-b499-2260d9da4992) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '356b6fbc-fccb-4b94-95cc-e6051cb7e306', 'volume_name': 'volume-356b6fbc-fccb-4b94-95cc-e6051cb7e306', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:39.042105 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (933dcc33-a9d9-42e5-b9d9-ee10f4839f3a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:39.068045 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (933dcc33-a9d9-42e5-b9d9-ee10f4839f3a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:39.072740 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (50d2985c-65e1-4765-8ea5-ff255b41729d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:39.073864 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Volume 356b6fbc-fccb-4b94-95cc-e6051cb7e306: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-356b6fbc-fccb-4b94-95cc-e6051cb7e306', 'volume_size': 1} Oct 01 21:04:39.074267 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] creating volume 'volume-356b6fbc-fccb-4b94-95cc-e6051cb7e306' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:04:39.095528 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:39.192388 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-3d726d9d-3132-42a2-ad92-1e0ef26376fc req-4f44ad4d-48a0-48bc-801c-4a149e6ea5f4 tempest-ServerRescueNegativeTestJSON-894078329 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:04:39.272669 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:04:39.273109 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:04:39.311922 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (50d2985c-65e1-4765-8ea5-ff255b41729d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-356b6fbc-fccb-4b94-95cc-e6051cb7e306', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:39.320935 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (492365ff-8550-4737-a99c-91dcbfae94d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:39.337112 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Volume volume-356b6fbc-fccb-4b94-95cc-e6051cb7e306 (356b6fbc-fccb-4b94-95cc-e6051cb7e306): created successfully Oct 01 21:04:39.341593 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (492365ff-8550-4737-a99c-91dcbfae94d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:39.355903 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Flow 'volume_create_manager' (e56d7a6b-94b4-4797-81ea-57f1d2bb0521) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:04:39.385379 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-43073971-f9e4-4616-b71c-330cffd9c224 tempest-VolumesBackupsV39Test-507479253 None] Created volume successfully. Oct 01 21:04:39.445392 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-85528416-4170-4ff9-8e75-7f6a7f6251a0 req-0f736389-852c-4bac-8166-7db9d92a6d82 tempest-VolumesSnapshotsTestJSON-740423316 None] Acquiring lock "cinder-5111d210-c9bc-4c91-8596-75901bb29ae6-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:04:39.479532 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-85528416-4170-4ff9-8e75-7f6a7f6251a0 req-0f736389-852c-4bac-8166-7db9d92a6d82 tempest-VolumesSnapshotsTestJSON-740423316 None] Lock "cinder-5111d210-c9bc-4c91-8596-75901bb29ae6-delete_snapshot" acquired by "delete_snapshot" :: waited 0.036s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:04:39.484451 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-85528416-4170-4ff9-8e75-7f6a7f6251a0 req-0f736389-852c-4bac-8166-7db9d92a6d82 tempest-VolumesSnapshotsTestJSON-740423316 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:40.278145 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=67,cinder:UPDATE=24,cinder:INSERT=5,cinder:DELETE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:40.470324 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [req-85528416-4170-4ff9-8e75-7f6a7f6251a0 req-0f736389-852c-4bac-8166-7db9d92a6d82 tempest-VolumesSnapshotsTestJSON-740423316 None] Created reservations ['87d73ce0-891f-45ff-9e82-a4eaa719c1f9', 'bc81d924-a223-49fa-b297-271f684b5263', '783d4309-fd41-449c-bf62-2b1f77f0d758', '56f457c5-351d-463e-be56-663f17cc8b31'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:04:40.541657 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d7af6b6b-844e-4a5a-af8a-670a9553540a tempest-VolumesBackupsV39Test-507479253 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-10-01T21:04:40Z,data_timestamp=2022-10-01T21:04:40Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Backup-723161801',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=248a641d-21de-46e6-9fee-e700229d8dfd,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='cd3526ab00f3485e98db7f02e57bea4d',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='477eb33574da473284d09e765825bcc6',volume_id=356b6fbc-fccb-4b94-95cc-e6051cb7e306) Oct 01 21:04:40.571535 ncdv-master-1 cinder-volume[133391]: INFO cinder.rpc [None req-d7af6b6b-844e-4a5a-af8a-670a9553540a tempest-VolumesBackupsV39Test-507479253 None] Automatically selected cinder-backup objects version 1.39 as minimum service version. Oct 01 21:04:40.612124 ncdv-master-1 cinder-volume[133391]: INFO cinder.rpc [None req-d7af6b6b-844e-4a5a-af8a-670a9553540a tempest-VolumesBackupsV39Test-507479253 None] Automatically selected cinder-backup RPC version 2.3 as minimum service version. Oct 01 21:04:40.625081 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.backup.rpcapi [None req-d7af6b6b-844e-4a5a-af8a-670a9553540a tempest-VolumesBackupsV39Test-507479253 None] continue_backup in rpcapi backup_id 248a641d-21de-46e6-9fee-e700229d8dfd {{(pid=133391) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Oct 01 21:04:40.692721 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-85528416-4170-4ff9-8e75-7f6a7f6251a0 req-0f736389-852c-4bac-8166-7db9d92a6d82 tempest-VolumesSnapshotsTestJSON-740423316 None] Delete snapshot completed successfully. Oct 01 21:04:40.699867 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-85528416-4170-4ff9-8e75-7f6a7f6251a0 req-0f736389-852c-4bac-8166-7db9d92a6d82 tempest-VolumesSnapshotsTestJSON-740423316 None] Lock "cinder-5111d210-c9bc-4c91-8596-75901bb29ae6-delete_snapshot" released by "delete_snapshot" :: held 1.221s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:04:40.918154 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-3d726d9d-3132-42a2-ad92-1e0ef26376fc req-4f44ad4d-48a0-48bc-801c-4a149e6ea5f4 tempest-ServerRescueNegativeTestJSON-894078329 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.727s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:04:40.919706 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3d726d9d-3132-42a2-ad92-1e0ef26376fc req-4f44ad4d-48a0-48bc-801c-4a149e6ea5f4 tempest-ServerRescueNegativeTestJSON-894078329 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1067c786-686b-41aa-b9ca-31ea24d10308', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '1067c786-686b-41aa-b9ca-31ea24d10308', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:04:40.972796 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-3d726d9d-3132-42a2-ad92-1e0ef26376fc req-4f44ad4d-48a0-48bc-801c-4a149e6ea5f4 tempest-ServerRescueNegativeTestJSON-894078329 None] Connection info returned from driver {'name': 'volumes/volume-1067c786-686b-41aa-b9ca-31ea24d10308', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '1067c786-686b-41aa-b9ca-31ea24d10308', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '7bfffbd6-e39a-4ef0-a6f3-7cfcc3753175'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:04:41.106034 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-3d726d9d-3132-42a2-ad92-1e0ef26376fc req-4f44ad4d-48a0-48bc-801c-4a149e6ea5f4 tempest-ServerRescueNegativeTestJSON-894078329 None] attachment_update completed successfully. Oct 01 21:04:42.051340 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] Acquiring lock "cinder-3155b986-3afe-4c50-83df-264e88b679e1-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:04:42.083243 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] Lock "cinder-3155b986-3afe-4c50-83df-264e88b679e1-delete_volume" acquired by "delete_volume" :: waited 0.031s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:04:42.151181 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:42.271450 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:04:42.272387 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] Volume volume-3155b986-3afe-4c50-83df-264e88b679e1 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:04:42.291912 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] deleting rbd volume volume-3155b986-3afe-4c50-83df-264e88b679e1 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:04:42.725793 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] Created reservations ['662281d6-dc6d-4ff5-977c-f778a1f7d504', 'bdd7658c-7427-42bf-976a-0763e6601db1', 'a224b906-2a40-4c8e-95dc-34b3a876fb88', '08041ff7-e3a0-49dd-89f0-27751c568c45'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:04:42.826971 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:42.903011 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:42.967169 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:04:42.977085 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] Deleted volume successfully. Oct 01 21:04:43.029204 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-3d165861-3550-4d0e-bbd5-f562e7e14cf4 req-6ee54c13-7749-4d7a-bb23-4d13c6ed5d44 tempest-VolumesSnapshotsTestJSON-740423316 None] Lock "cinder-3155b986-3afe-4c50-83df-264e88b679e1-delete_volume" released by "delete_volume" :: held 0.945s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:04:50.014282 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-d7af6b6b-844e-4a5a-af8a-670a9553540a tempest-VolumesBackupsV39Test-507479253 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:04:51.607826 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-d7af6b6b-844e-4a5a-af8a-670a9553540a tempest-VolumesBackupsV39Test-507479253 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.593s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:04:51.609301 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d7af6b6b-844e-4a5a-af8a-670a9553540a tempest-VolumesBackupsV39Test-507479253 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-356b6fbc-fccb-4b94-95cc-e6051cb7e306', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '356b6fbc-fccb-4b94-95cc-e6051cb7e306', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:04:51.651041 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d7af6b6b-844e-4a5a-af8a-670a9553540a tempest-VolumesBackupsV39Test-507479253 None] Initialize volume connection completed successfully. Oct 01 21:04:52.077192 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:04:52.109655 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Flow 'volume_create_manager' (5d463ee1-09b6-4bb6-a462-e7d587130fca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:04:52.117215 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f55d447a-025a-42de-82c4-518ad33e939c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:52.137650 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f55d447a-025a-42de-82c4-518ad33e939c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:04:52Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTestJSON-Volume-684926199',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8679355b-edc3-4fe0-ad65-ecd888d68e94,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='3196d4aa5c2040a2b114b8b21352e4c2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:04:52Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:04:52Z,use_quota=True,user_id='d042705af4284cca8741304d690ae343',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:52.143702 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (741c4b9f-d097-4fe6-952c-8e785d3aac0c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:52.148278 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (741c4b9f-d097-4fe6-952c-8e785d3aac0c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:52.153228 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bf7b0850-171c-4410-abdb-77d85e2f0a64) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:52.157405 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bf7b0850-171c-4410-abdb-77d85e2f0a64) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8679355b-edc3-4fe0-ad65-ecd888d68e94', 'volume_name': 'volume-8679355b-edc3-4fe0-ad65-ecd888d68e94', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:52.165151 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (29db6a59-43b9-4343-8a75-b966ac93ac92) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:52.169300 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (29db6a59-43b9-4343-8a75-b966ac93ac92) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:52.174388 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (76cbf172-1e08-4452-9ed8-bf9d85e8d106) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:52.175766 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Volume 8679355b-edc3-4fe0-ad65-ecd888d68e94: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8679355b-edc3-4fe0-ad65-ecd888d68e94', 'volume_size': 1} Oct 01 21:04:52.176573 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] creating volume 'volume-8679355b-edc3-4fe0-ad65-ecd888d68e94' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:04:52.178186 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:52.287617 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:04:52.288292 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:04:52.292887 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (76cbf172-1e08-4452-9ed8-bf9d85e8d106) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8679355b-edc3-4fe0-ad65-ecd888d68e94', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:52.301171 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (de94cc13-a092-4eac-ba63-01093dfa5cb3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:52.326649 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Volume volume-8679355b-edc3-4fe0-ad65-ecd888d68e94 (8679355b-edc3-4fe0-ad65-ecd888d68e94): created successfully Oct 01 21:04:52.334676 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (de94cc13-a092-4eac-ba63-01093dfa5cb3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:52.342161 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Flow 'volume_create_manager' (5d463ee1-09b6-4bb6-a462-e7d587130fca) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:04:52.354186 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-fe2853f8-f387-483c-ba88-642832108f1f req-84218660-30dd-4334-94db-c6dd5b19d5e9 tempest-VolumesGetTestJSON-2063573397 None] Created volume successfully. Oct 01 21:04:52.953267 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] Acquiring lock "cinder-8679355b-edc3-4fe0-ad65-ecd888d68e94-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:04:52.974960 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] Lock "cinder-8679355b-edc3-4fe0-ad65-ecd888d68e94-delete_volume" acquired by "delete_volume" :: waited 0.021s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:04:53.051361 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:53.145216 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:04:53.145670 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] Volume volume-8679355b-edc3-4fe0-ad65-ecd888d68e94 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:04:53.154052 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] deleting rbd volume volume-8679355b-edc3-4fe0-ad65-ecd888d68e94 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:04:53.605285 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] Created reservations ['510bda96-eba2-4270-b019-edfdc517eead', 'c3c45830-f73c-45df-8d31-8fb7f568469d', 'a304abaf-35aa-4148-987e-cc1302cf0e0a', 'abba255e-d5a0-4ac1-87e5-eb95e071b6d6'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:04:53.680589 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:53.762817 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:53.862271 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:04:53.892608 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] Deleted volume successfully. Oct 01 21:04:53.976850 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-37796524-c63d-4f8f-99c2-61e1955e61a8 req-60b46f83-9d7b-4859-a195-a76a5fcdd946 tempest-VolumesGetTestJSON-2063573397 None] Lock "cinder-8679355b-edc3-4fe0-ad65-ecd888d68e94-delete_volume" released by "delete_volume" :: held 1.002s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:04:55.630935 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=62,cinder:UPDATE=31,cinder:INSERT=6,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:55.651682 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d7af6b6b-844e-4a5a-af8a-670a9553540a tempest-VolumesBackupsV39Test-507479253 None] Terminate volume connection completed successfully. Oct 01 21:04:55.685099 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d7af6b6b-844e-4a5a-af8a-670a9553540a tempest-VolumesBackupsV39Test-507479253 None] Remove volume export completed successfully. Oct 01 21:04:56.092878 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:04:56.140398 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Flow 'volume_create_manager' (f96c06e5-f59b-45bf-9272-b888827fc07d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:04:56.152268 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2ba22664-1ff4-434e-87a1-9336b06ad218) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:56.192281 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2ba22664-1ff4-434e-87a1-9336b06ad218) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:04:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-82573255',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=10008470-ec42-45af-91f0-d055c3127e46,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0bf01814edfe46d5b1dd62f42c2fa45d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:04:56Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:04:56Z,use_quota=True,user_id='9b5f90f4ec8b4b5cb4b64a2b99ca9beb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:56.197918 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0df55cdd-03e3-4939-9265-14a4022cc2d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:56.203640 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0df55cdd-03e3-4939-9265-14a4022cc2d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:56.209021 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (48949e61-c39a-4905-98aa-de60b8330a0a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:56.419570 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (48949e61-c39a-4905-98aa-de60b8330a0a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '10008470-ec42-45af-91f0-d055c3127e46', 'volume_name': 'volume-10008470-ec42-45af-91f0-d055c3127e46', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:56.432941 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cade0853-0446-4724-842a-fa871f538f34) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:56.446458 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cade0853-0446-4724-842a-fa871f538f34) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:56.459133 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3f5c8729-414f-4f28-98f2-ceae413c0b84) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:56.460610 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Volume 10008470-ec42-45af-91f0-d055c3127e46: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-10008470-ec42-45af-91f0-d055c3127e46', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:04:56.461222 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Cloning 10008470-ec42-45af-91f0-d055c3127e46 from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:04:56.461982 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:04:56.481324 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.019s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:04:56.502560 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 4, 56, 491224)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:04:56.503389 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:04:56.504071 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:04:56.509539 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.028s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:04:56.511025 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:04:56.533904 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 4, 56, 520801)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:04:56.534691 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:04:56.535552 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:04:56.564825 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:56.713169 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] creating snapshot='volume-10008470-ec42-45af-91f0-d055c3127e46.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:04:57.450392 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-10008470-ec42-45af-91f0-d055c3127e46.clone_snap' to 'volume-10008470-ec42-45af-91f0-d055c3127e46' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:04:57.692243 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:04:57.695556 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:04:57.696188 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:04:57.737586 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:04:57.738221 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Marking volume 10008470-ec42-45af-91f0-d055c3127e46 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:04:57.757862 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to 10008470-ec42-45af-91f0-d055c3127e46. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:04:57.758566 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Creating volume glance metadata for volume 10008470-ec42-45af-91f0-d055c3127e46 backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:04:57.762371 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=66,cinder:INSERT=6,cinder:UPDATE=26,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:04:57.908523 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3f5c8729-414f-4f28-98f2-ceae413c0b84) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-10008470-ec42-45af-91f0-d055c3127e46', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:57.914104 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e2308109-5ecb-4ebe-97fc-ab94499eb013) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:57.945262 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Volume volume-10008470-ec42-45af-91f0-d055c3127e46 (10008470-ec42-45af-91f0-d055c3127e46): created successfully Oct 01 21:04:57.951122 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e2308109-5ecb-4ebe-97fc-ab94499eb013) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:57.970648 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Flow 'volume_create_manager' (f96c06e5-f59b-45bf-9272-b888827fc07d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:04:57.987123 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0da8723c-9ec1-477c-b46f-8023c0e553d4 tempest-TaggedBootDevicesTest-874492856 None] Created volume successfully. Oct 01 21:04:58.965435 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-2ff12d0b-7e4e-4aad-b40d-6e569f0d22bd tempest-VolumesBackupsV39Test-507479253 None] Acquiring lock "cinder-356b6fbc-fccb-4b94-95cc-e6051cb7e306-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:04:59.005758 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-2ff12d0b-7e4e-4aad-b40d-6e569f0d22bd tempest-VolumesBackupsV39Test-507479253 None] Lock "cinder-356b6fbc-fccb-4b94-95cc-e6051cb7e306-delete_volume" acquired by "delete_volume" :: waited 0.040s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:04:59.080426 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:04:59.144234 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Flow 'volume_create_manager' (3e07f1b0-5f64-43b1-9567-1ccde39401d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:04:59.152763 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1dbe8635-cd81-4500-b3a1-4ca380467ff6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:59.232666 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1dbe8635-cd81-4500-b3a1-4ca380467ff6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:04:58Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-1717098085',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=191e827c-2032-49f3-ba21-01dce9c29e11,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0bf01814edfe46d5b1dd62f42c2fa45d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:04:59Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:04:59Z,use_quota=True,user_id='9b5f90f4ec8b4b5cb4b64a2b99ca9beb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:59.249953 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (901e9156-53a9-42f1-a22b-ddebf8ecea99) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:59.254859 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (901e9156-53a9-42f1-a22b-ddebf8ecea99) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:59.265094 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2b42806c-d718-4c8c-9323-fc05898f94ed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:59.276199 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2b42806c-d718-4c8c-9323-fc05898f94ed) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '191e827c-2032-49f3-ba21-01dce9c29e11', 'volume_name': 'volume-191e827c-2032-49f3-ba21-01dce9c29e11', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:59.287859 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (962b793e-5c7c-4147-ace2-c16142e2107a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:59.292142 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (962b793e-5c7c-4147-ace2-c16142e2107a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:04:59.312310 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9708aed2-cdb1-44ee-b481-8c379745711f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:04:59.313998 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Volume 191e827c-2032-49f3-ba21-01dce9c29e11: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-191e827c-2032-49f3-ba21-01dce9c29e11', 'volume_size': 1} Oct 01 21:04:59.314393 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] creating volume 'volume-191e827c-2032-49f3-ba21-01dce9c29e11' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:04:59.315195 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:04:59.350082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2ff12d0b-7e4e-4aad-b40d-6e569f0d22bd tempest-VolumesBackupsV39Test-507479253 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:00.474284 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2ff12d0b-7e4e-4aad-b40d-6e569f0d22bd tempest-VolumesBackupsV39Test-507479253 None] Volume volume-356b6fbc-fccb-4b94-95cc-e6051cb7e306 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:05:00.522462 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2ff12d0b-7e4e-4aad-b40d-6e569f0d22bd tempest-VolumesBackupsV39Test-507479253 None] deleting rbd volume volume-356b6fbc-fccb-4b94-95cc-e6051cb7e306 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:05:00.532375 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:05:00.532375 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:05:00.540240 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9708aed2-cdb1-44ee-b481-8c379745711f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-191e827c-2032-49f3-ba21-01dce9c29e11', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:00.552279 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5e4329d2-fb70-40f7-b304-f5063da60680) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:00.596482 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Volume volume-191e827c-2032-49f3-ba21-01dce9c29e11 (191e827c-2032-49f3-ba21-01dce9c29e11): created successfully Oct 01 21:05:00.600998 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5e4329d2-fb70-40f7-b304-f5063da60680) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:00.627933 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Flow 'volume_create_manager' (3e07f1b0-5f64-43b1-9567-1ccde39401d0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:05:00.696655 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8139b090-7437-4649-b7e8-b646485d578f tempest-TaggedBootDevicesTest-874492856 None] Created volume successfully. Oct 01 21:05:00.982894 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-2ff12d0b-7e4e-4aad-b40d-6e569f0d22bd tempest-VolumesBackupsV39Test-507479253 None] Created reservations ['6505b4f7-385c-4799-8d7d-e4df86d25053', '2cc6bb80-c401-496d-995b-9b9ba1410df4', '8bbe71ed-89d8-4e14-a67a-e3eb46bee6a5', '93bbdb75-2788-4431-b7a9-a487d9cdc857'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:05:01.124465 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2ff12d0b-7e4e-4aad-b40d-6e569f0d22bd tempest-VolumesBackupsV39Test-507479253 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:01.215685 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2ff12d0b-7e4e-4aad-b40d-6e569f0d22bd tempest-VolumesBackupsV39Test-507479253 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:01.284702 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-2ff12d0b-7e4e-4aad-b40d-6e569f0d22bd tempest-VolumesBackupsV39Test-507479253 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:05:01.304563 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-2ff12d0b-7e4e-4aad-b40d-6e569f0d22bd tempest-VolumesBackupsV39Test-507479253 None] Deleted volume successfully. Oct 01 21:05:01.379870 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-2ff12d0b-7e4e-4aad-b40d-6e569f0d22bd tempest-VolumesBackupsV39Test-507479253 None] Lock "cinder-356b6fbc-fccb-4b94-95cc-e6051cb7e306-delete_volume" released by "delete_volume" :: held 2.374s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:05:02.106905 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:05:02.133281 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Flow 'volume_create_manager' (88f898d0-2677-474f-ab27-21cc618dfa3e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:05:02.139791 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cf26d779-6c6b-41bf-9dcf-df65da3135ae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:02.177166 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cf26d779-6c6b-41bf-9dcf-df65da3135ae) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:05:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-870378399',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5f3eb6ba-d61b-4c3c-8018-5709f993e46d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0bf01814edfe46d5b1dd62f42c2fa45d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:05:02Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:05:02Z,use_quota=True,user_id='9b5f90f4ec8b4b5cb4b64a2b99ca9beb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:02.188178 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (76f14c30-bfae-4794-b060-3dd60a942398) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:02.199342 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (76f14c30-bfae-4794-b060-3dd60a942398) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:02.216773 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a87da6be-4ec0-4507-a870-071e3223bbdd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:02.228323 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a87da6be-4ec0-4507-a870-071e3223bbdd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5f3eb6ba-d61b-4c3c-8018-5709f993e46d', 'volume_name': 'volume-5f3eb6ba-d61b-4c3c-8018-5709f993e46d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:02.237449 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6d097697-bc9a-4df9-bf84-d4bbbcd2cb3d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:02.245727 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6d097697-bc9a-4df9-bf84-d4bbbcd2cb3d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:02.267227 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (de845ff7-e34b-4e60-9aa4-20d34a649385) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:02.269515 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Volume 5f3eb6ba-d61b-4c3c-8018-5709f993e46d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5f3eb6ba-d61b-4c3c-8018-5709f993e46d', 'volume_size': 1} Oct 01 21:05:02.272813 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] creating volume 'volume-5f3eb6ba-d61b-4c3c-8018-5709f993e46d' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:05:02.273870 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:02.439645 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:05:02.440138 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:05:02.443893 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (de845ff7-e34b-4e60-9aa4-20d34a649385) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5f3eb6ba-d61b-4c3c-8018-5709f993e46d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:02.449321 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fbb65a58-ffce-4b48-80d8-db394c526fc8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:02.493697 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Volume volume-5f3eb6ba-d61b-4c3c-8018-5709f993e46d (5f3eb6ba-d61b-4c3c-8018-5709f993e46d): created successfully Oct 01 21:05:02.512351 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fbb65a58-ffce-4b48-80d8-db394c526fc8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:02.535255 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Flow 'volume_create_manager' (88f898d0-2677-474f-ab27-21cc618dfa3e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:05:02.567405 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-17cc6c4d-6edf-46cb-8b9c-dbb5c6314f67 tempest-TaggedBootDevicesTest-874492856 None] Created volume successfully. Oct 01 21:05:07.957454 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:05:08.001479 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Flow 'volume_create_manager' (0af8babd-1e60-4286-933f-b4bcb67bffdc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:05:08.016272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c04eb4c2-affe-4c4d-bcd4-54fe8d86448c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:08.060408 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c04eb4c2-affe-4c4d-bcd4-54fe8d86448c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:05:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1203781845',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=0cc47213-d39d-481b-bbe2-5ecf1101a690,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d591215350fc4c34939709505679a996',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:05:08Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:05:08Z,use_quota=True,user_id='4b540aab3baf4bf7b28eb8a7e9c37eb0',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:08.070126 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7ef83dc9-ad87-49e5-87be-cd03f9a770bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:08.083929 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7ef83dc9-ad87-49e5-87be-cd03f9a770bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:08.099082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (085e421e-022e-4179-a8ce-62388bad3bfc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:08.260439 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (085e421e-022e-4179-a8ce-62388bad3bfc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '0cc47213-d39d-481b-bbe2-5ecf1101a690', 'volume_name': 'volume-0cc47213-d39d-481b-bbe2-5ecf1101a690', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:08.274811 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2112cea8-717b-450a-88ff-e4ac3b77b377) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:08.284182 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2112cea8-717b-450a-88ff-e4ac3b77b377) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:08.304655 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fa8c82d1-6847-4824-ab36-b997cd99608a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:08.306359 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Volume 0cc47213-d39d-481b-bbe2-5ecf1101a690: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-0cc47213-d39d-481b-bbe2-5ecf1101a690', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:05:08.308419 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Cloning 0cc47213-d39d-481b-bbe2-5ecf1101a690 from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:05:08.310875 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:05:08.328766 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.018s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:05:08.356224 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 5, 8, 344087)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:05:08.358987 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:05:08.362561 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:05:08.370771 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.042s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:05:08.372106 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:05:08.385440 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 5, 8, 382541)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:05:08.387591 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:05:08.388239 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:05:08.405808 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:08.520765 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] creating snapshot='volume-0cc47213-d39d-481b-bbe2-5ecf1101a690.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:05:09.227132 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-0cc47213-d39d-481b-bbe2-5ecf1101a690.clone_snap' to 'volume-0cc47213-d39d-481b-bbe2-5ecf1101a690' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:05:09.472545 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:05:09.474067 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:05:09.474443 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:05:09.516442 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:05:09.517093 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Marking volume 0cc47213-d39d-481b-bbe2-5ecf1101a690 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:05:09.529410 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to 0cc47213-d39d-481b-bbe2-5ecf1101a690. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:05:09.529884 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Creating volume glance metadata for volume 0cc47213-d39d-481b-bbe2-5ecf1101a690 backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:05:09.604937 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=62,cinder:UPDATE=20,cinder:INSERT=15,cinder:DELETE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:09.619681 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fa8c82d1-6847-4824-ab36-b997cd99608a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-0cc47213-d39d-481b-bbe2-5ecf1101a690', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:09.636776 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d63f332c-1c50-4fe2-9fd3-61215500251e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:09.651717 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Volume volume-0cc47213-d39d-481b-bbe2-5ecf1101a690 (0cc47213-d39d-481b-bbe2-5ecf1101a690): created successfully Oct 01 21:05:09.659716 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d63f332c-1c50-4fe2-9fd3-61215500251e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:09.678927 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Flow 'volume_create_manager' (0af8babd-1e60-4286-933f-b4bcb67bffdc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:05:09.700547 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-fef6804e-343b-4aba-a083-b353a0ac66ac tempest-VolumesCloneTest-68372025 None] Created volume successfully. Oct 01 21:05:10.825996 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:05:10.903077 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Flow 'volume_create_manager' (474e954d-c919-463e-8826-03fb273d7b87) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:05:10.911307 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2d993490-db08-4a13-af31-765dab97750d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:10.938976 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2d993490-db08-4a13-af31-765dab97750d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:05:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-2077550065',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c8c1cf48-de61-4873-9361-9f9e30560bf4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d591215350fc4c34939709505679a996',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:05:11Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=0cc47213-d39d-481b-bbe2-5ecf1101a690,status='creating',terminated_at=None,updated_at=2022-10-01T21:05:11Z,use_quota=True,user_id='4b540aab3baf4bf7b28eb8a7e9c37eb0',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:10.969748 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b790a143-f8a7-4c16-a157-6df7e40a06f0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:10.998354 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b790a143-f8a7-4c16-a157-6df7e40a06f0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:11.009223 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (05466e58-8fd1-4aee-9872-61cc69e6fecf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:11.053158 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (05466e58-8fd1-4aee-9872-61cc69e6fecf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': 'c8c1cf48-de61-4873-9361-9f9e30560bf4', 'volume_name': 'volume-c8c1cf48-de61-4873-9361-9f9e30560bf4', 'volume_size': 1, 'source_volid': '0cc47213-d39d-481b-bbe2-5ecf1101a690', 'source_volstatus': 'available'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:11.065460 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9f4888b5-da71-4adc-a63c-6f65314a915e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:11.070830 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9f4888b5-da71-4adc-a63c-6f65314a915e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:11.091879 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f50b7570-5e6b-444e-b7b5-a1ed65a2148d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:11.093114 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Volume c8c1cf48-de61-4873-9361-9f9e30560bf4: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-c8c1cf48-de61-4873-9361-9f9e30560bf4', 'volume_size': 1, 'source_volid': '0cc47213-d39d-481b-bbe2-5ecf1101a690', 'source_volstatus': 'available'} Oct 01 21:05:11.115325 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:11.280116 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] creating snapshot='volume-c8c1cf48-de61-4873-9361-9f9e30560bf4.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:05:12.297456 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] cloning 'volume-0cc47213-d39d-481b-bbe2-5ecf1101a690@volume-c8c1cf48-de61-4873-9361-9f9e30560bf4.clone_snap' to 'volume-c8c1cf48-de61-4873-9361-9f9e30560bf4' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:05:12.625688 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:05:12.627557 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:05:12.628201 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:05:12.672137 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:05:12.672942 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Marking volume c8c1cf48-de61-4873-9361-9f9e30560bf4 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:05:12.687189 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Copying metadata from source volume 0cc47213-d39d-481b-bbe2-5ecf1101a690 to c8c1cf48-de61-4873-9361-9f9e30560bf4. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:440}} Oct 01 21:05:12.793044 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f50b7570-5e6b-444e-b7b5-a1ed65a2148d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c8c1cf48-de61-4873-9361-9f9e30560bf4', 'volume_size': 1, 'source_volid': '0cc47213-d39d-481b-bbe2-5ecf1101a690', 'source_volstatus': 'available'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:12.810279 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9a3b41e5-a49b-4568-9ef9-b578bfbfc87a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:12.839443 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Volume volume-c8c1cf48-de61-4873-9361-9f9e30560bf4 (c8c1cf48-de61-4873-9361-9f9e30560bf4): created successfully Oct 01 21:05:12.848136 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9a3b41e5-a49b-4568-9ef9-b578bfbfc87a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:12.863355 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Flow 'volume_create_manager' (474e954d-c919-463e-8826-03fb273d7b87) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:05:12.893062 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-640a1a34-a0c6-477d-a8cc-7165bc6974d1 tempest-VolumesCloneTest-68372025 None] Created volume successfully. Oct 01 21:05:13.836230 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:05:13.913915 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Flow 'volume_create_manager' (f0bf030f-5607-4732-a8e8-6da954eabf59) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:05:13.937366 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2849c5c4-64de-4c48-82ee-471ec760bb31) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:14.012203 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2849c5c4-64de-4c48-82ee-471ec760bb31) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:05:13Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-772182299',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c0872ce8-e66d-4925-9867-40b48f06232d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d591215350fc4c34939709505679a996',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:05:14Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:05:14Z,use_quota=True,user_id='4b540aab3baf4bf7b28eb8a7e9c37eb0',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:14.022430 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c20e7ab0-607a-4385-850a-b3fc90ffb13d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:14.068343 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c20e7ab0-607a-4385-850a-b3fc90ffb13d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:14.105910 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (723be59e-c665-4b61-ab6a-0900e62aed3e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:14.119051 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (723be59e-c665-4b61-ab6a-0900e62aed3e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c0872ce8-e66d-4925-9867-40b48f06232d', 'volume_name': 'volume-c0872ce8-e66d-4925-9867-40b48f06232d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:14.124704 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8a7a19aa-270f-4b79-baf3-7b7fa91a5698) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:14.133203 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8a7a19aa-270f-4b79-baf3-7b7fa91a5698) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:14.151872 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (467595ce-eada-4a5d-9354-82f0d3fe89b6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:14.153275 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Volume c0872ce8-e66d-4925-9867-40b48f06232d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c0872ce8-e66d-4925-9867-40b48f06232d', 'volume_size': 1} Oct 01 21:05:14.153846 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] creating volume 'volume-c0872ce8-e66d-4925-9867-40b48f06232d' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:05:14.154709 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:14.295779 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:05:14.296529 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:05:14.302602 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (467595ce-eada-4a5d-9354-82f0d3fe89b6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c0872ce8-e66d-4925-9867-40b48f06232d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:14.322338 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (386ed8b1-b642-4260-a426-71e80a7bfad9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:14.363025 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Volume volume-c0872ce8-e66d-4925-9867-40b48f06232d (c0872ce8-e66d-4925-9867-40b48f06232d): created successfully Oct 01 21:05:14.378456 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (386ed8b1-b642-4260-a426-71e80a7bfad9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:14.415721 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Flow 'volume_create_manager' (f0bf030f-5607-4732-a8e8-6da954eabf59) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:05:14.441040 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4b05f993-74ad-4945-a493-55156974cf33 tempest-VolumesCloneTest-68372025 None] Created volume successfully. Oct 01 21:05:15.937054 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:05:16.061293 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Flow 'volume_create_manager' (151d97aa-b61a-474a-b4cf-140c9ec378ad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:05:16.084746 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5bed66aa-c9ea-4cf8-ab47-6377c2c57218) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:16.148727 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5bed66aa-c9ea-4cf8-ab47-6377c2c57218) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:05:15Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1528861093',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cb5c8572-edd1-4802-9fe7-71775ae123dc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d591215350fc4c34939709505679a996',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:05:16Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=c0872ce8-e66d-4925-9867-40b48f06232d,status='creating',terminated_at=None,updated_at=2022-10-01T21:05:16Z,use_quota=True,user_id='4b540aab3baf4bf7b28eb8a7e9c37eb0',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:16.172829 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c4c2b3c0-0a2c-45ff-b048-99647c994323) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:16.184812 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c4c2b3c0-0a2c-45ff-b048-99647c994323) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:16.194307 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8da5f1b5-035f-44d1-a236-5724e61c0ccd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:16.245121 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8da5f1b5-035f-44d1-a236-5724e61c0ccd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': 'cb5c8572-edd1-4802-9fe7-71775ae123dc', 'volume_name': 'volume-cb5c8572-edd1-4802-9fe7-71775ae123dc', 'volume_size': 2, 'source_volid': 'c0872ce8-e66d-4925-9867-40b48f06232d', 'source_volstatus': 'available'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:16.261485 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (22481561-5064-4047-b539-1357426ceee9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:16.273559 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (22481561-5064-4047-b539-1357426ceee9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:16.289179 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0f3ef46f-4f2b-4382-8dc4-4eef35f75459) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:16.293490 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Volume cb5c8572-edd1-4802-9fe7-71775ae123dc: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-cb5c8572-edd1-4802-9fe7-71775ae123dc', 'volume_size': 2, 'source_volid': 'c0872ce8-e66d-4925-9867-40b48f06232d', 'source_volstatus': 'available'} Oct 01 21:05:16.316725 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:16.386762 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] creating snapshot='volume-cb5c8572-edd1-4802-9fe7-71775ae123dc.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:05:17.409816 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] cloning 'volume-c0872ce8-e66d-4925-9867-40b48f06232d@volume-cb5c8572-edd1-4802-9fe7-71775ae123dc.clone_snap' to 'volume-cb5c8572-edd1-4802-9fe7-71775ae123dc' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:05:17.581793 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Volume volume-c0872ce8-e66d-4925-9867-40b48f06232d is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:05:17.583023 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:05:17.585185 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:05:17.613211 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] resize volume 'volume-cb5c8572-edd1-4802-9fe7-71775ae123dc' from 1 to 2 {{(pid=133391) _extend_if_required /opt/stack/cinder/cinder/volume/drivers/rbd.py:812}} Oct 01 21:05:17.614224 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:17.846265 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:05:17.850231 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0f3ef46f-4f2b-4382-8dc4-4eef35f75459) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cb5c8572-edd1-4802-9fe7-71775ae123dc', 'volume_size': 2, 'source_volid': 'c0872ce8-e66d-4925-9867-40b48f06232d', 'source_volstatus': 'available'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:17.861426 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5c8853c0-199c-478f-ac59-38fc757c622c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:05:17.876218 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Volume volume-cb5c8572-edd1-4802-9fe7-71775ae123dc (cb5c8572-edd1-4802-9fe7-71775ae123dc): created successfully Oct 01 21:05:17.884082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5c8853c0-199c-478f-ac59-38fc757c622c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:05:17.891998 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Flow 'volume_create_manager' (151d97aa-b61a-474a-b4cf-140c9ec378ad) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:05:17.931668 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a107778f-532f-45c9-9e23-7a0d38842153 tempest-VolumesCloneTest-68372025 None] Created volume successfully. Oct 01 21:05:18.305205 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] Acquiring lock "cinder-cb5c8572-edd1-4802-9fe7-71775ae123dc-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:05:18.319157 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] Lock "cinder-cb5c8572-edd1-4802-9fe7-71775ae123dc-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:05:18.414286 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:18.567611 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:05:18.580511 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] deleting rbd volume volume-cb5c8572-edd1-4802-9fe7-71775ae123dc {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:05:18.947717 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:05:18.974255 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] Volume volume-c0872ce8-e66d-4925-9867-40b48f06232d is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:05:18.975187 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] deleting parent snapshot volume-cb5c8572-edd1-4802-9fe7-71775ae123dc.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:05:19.497862 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:INSERT=20,cinder:SELECT=58,cinder:UPDATE=20,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:19.561062 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] Created reservations ['61b8ecee-8e15-4dbf-b63e-eabff67dfe2b', 'cf6393fb-b0ca-4f4c-8e78-904660cca473', '5f4126fd-6419-499d-b3d4-c122044036c2', '3493c370-28c1-4b52-8b9c-ee87520c3b34'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:05:19.662612 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:19.763410 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:19.857603 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:05:19.871357 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] Deleted volume successfully. Oct 01 21:05:19.887790 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-706f4244-1779-4e00-8c79-03342f925f17 tempest-VolumesCloneTest-68372025 None] Lock "cinder-cb5c8572-edd1-4802-9fe7-71775ae123dc-delete_volume" released by "delete_volume" :: held 1.570s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:05:20.719623 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] Acquiring lock "cinder-c0872ce8-e66d-4925-9867-40b48f06232d-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:05:20.759574 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] Lock "cinder-c0872ce8-e66d-4925-9867-40b48f06232d-delete_volume" acquired by "delete_volume" :: waited 0.040s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:05:20.809113 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:20.914150 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:05:20.915089 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] Volume volume-c0872ce8-e66d-4925-9867-40b48f06232d is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:05:20.922342 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] deleting rbd volume volume-c0872ce8-e66d-4925-9867-40b48f06232d {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:05:21.304071 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] Created reservations ['2f64ea76-533d-4872-87dd-b9c8623f605a', '89811943-4f73-43fc-949b-53f4b53fd275', 'a8047606-9ca5-4e9a-bfa0-87566498fb75', 'f8239d7b-196c-4b60-a335-75dbd992f100'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:05:21.493630 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:21.567994 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:21.630453 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:05:21.635021 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] Deleted volume successfully. Oct 01 21:05:21.659965 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-73577c5e-ce6e-4d33-93cd-9bc3fcc53871 tempest-VolumesCloneTest-68372025 None] Lock "cinder-c0872ce8-e66d-4925-9867-40b48f06232d-delete_volume" released by "delete_volume" :: held 0.900s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:05:22.135783 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] Acquiring lock "cinder-c8c1cf48-de61-4873-9361-9f9e30560bf4-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:05:22.149318 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] Lock "cinder-c8c1cf48-de61-4873-9361-9f9e30560bf4-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:05:22.245708 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:22.380815 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:05:22.392055 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] deleting rbd volume volume-c8c1cf48-de61-4873-9361-9f9e30560bf4 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:05:22.776150 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:05:22.829392 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] deleting parent snapshot volume-c8c1cf48-de61-4873-9361-9f9e30560bf4.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:05:23.385337 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=61,cinder:UPDATE=25,cinder:INSERT=11,cinder:DELETE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:23.437129 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] Created reservations ['9ab2d96c-814d-49ec-8e65-3c4671e6b599', 'ea285340-5c73-4c7d-8c30-666875411cbc', 'a59285fd-0f26-4cf5-9d04-155965a730c2', '3e29d95b-047d-45a9-a871-94263a2ef33a'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:05:23.527675 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:23.633056 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:23.715704 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:05:23.726063 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:05:23.726931 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:23.809891 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] Deleted volume successfully. Oct 01 21:05:23.821192 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:23.882363 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:05:23.914682 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fc6abbd3-a9ca-4430-9936-9051fc83a9d4 tempest-VolumesCloneTest-68372025 None] Lock "cinder-c8c1cf48-de61-4873-9361-9f9e30560bf4-delete_volume" released by "delete_volume" :: held 1.765s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:05:24.445965 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] Acquiring lock "cinder-0cc47213-d39d-481b-bbe2-5ecf1101a690-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:05:24.469178 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] Lock "cinder-0cc47213-d39d-481b-bbe2-5ecf1101a690-delete_volume" acquired by "delete_volume" :: waited 0.021s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:05:24.540875 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:24.648371 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:05:24.657868 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] deleting rbd volume volume-0cc47213-d39d-481b-bbe2-5ecf1101a690 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:05:24.869461 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:05:24.903727 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] deleting parent snapshot volume-0cc47213-d39d-481b-bbe2-5ecf1101a690.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:05:25.296666 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] Created reservations ['8249a2f5-2077-460b-a666-d948af32976e', '5688a886-6f53-4699-af0e-901507e7b539', 'b3965835-c785-49d6-ba46-6865015278b9', 'f802a7a5-9465-4ce3-810f-6b59264c256d'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:05:25.384172 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:25.452626 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:05:25.494767 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:05:25.500140 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] Deleted volume successfully. Oct 01 21:05:25.518346 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4afadd3e-7273-459f-98d1-61b67e867fe9 tempest-VolumesCloneTest-68372025 None] Lock "cinder-0cc47213-d39d-481b-bbe2-5ecf1101a690-delete_volume" released by "delete_volume" :: held 1.051s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:05:28.925961 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-13b6750e-8163-4c3d-9fa6-3d1a5c18e583 tempest-TaggedBootDevicesTest-874492856 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:05:30.151244 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-13b6750e-8163-4c3d-9fa6-3d1a5c18e583 tempest-TaggedBootDevicesTest-874492856 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.225s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:05:30.152797 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-13b6750e-8163-4c3d-9fa6-3d1a5c18e583 tempest-TaggedBootDevicesTest-874492856 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-10008470-ec42-45af-91f0-d055c3127e46', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '10008470-ec42-45af-91f0-d055c3127e46', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:05:30.207785 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-13b6750e-8163-4c3d-9fa6-3d1a5c18e583 tempest-TaggedBootDevicesTest-874492856 None] Connection info returned from driver {'name': 'volumes/volume-10008470-ec42-45af-91f0-d055c3127e46', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '10008470-ec42-45af-91f0-d055c3127e46', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '67b8a6b4-6deb-4978-8d60-6840a0a81807'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:05:30.310263 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-13b6750e-8163-4c3d-9fa6-3d1a5c18e583 tempest-TaggedBootDevicesTest-874492856 None] attachment_update completed successfully. Oct 01 21:05:31.332944 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=35,cinder:DELETE=4,cinder:SELECT=53,cinder:INSERT=8 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:31.345401 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-11870206-5a74-48bf-b212-203e9a1eb597 tempest-TaggedBootDevicesTest-874492856 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:05:32.765725 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-11870206-5a74-48bf-b212-203e9a1eb597 tempest-TaggedBootDevicesTest-874492856 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.420s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:05:32.767422 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-11870206-5a74-48bf-b212-203e9a1eb597 tempest-TaggedBootDevicesTest-874492856 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-191e827c-2032-49f3-ba21-01dce9c29e11', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '191e827c-2032-49f3-ba21-01dce9c29e11', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:05:32.834164 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-11870206-5a74-48bf-b212-203e9a1eb597 tempest-TaggedBootDevicesTest-874492856 None] Connection info returned from driver {'name': 'volumes/volume-191e827c-2032-49f3-ba21-01dce9c29e11', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '191e827c-2032-49f3-ba21-01dce9c29e11', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'f383a399-c27d-4f0d-9959-c87686a9c836'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:05:32.948636 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-11870206-5a74-48bf-b212-203e9a1eb597 tempest-TaggedBootDevicesTest-874492856 None] attachment_update completed successfully. Oct 01 21:05:33.492424 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-0976e71b-92d5-43ee-9c98-44a3f941cadf tempest-TaggedBootDevicesTest-874492856 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:05:34.896051 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-0976e71b-92d5-43ee-9c98-44a3f941cadf tempest-TaggedBootDevicesTest-874492856 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.403s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:05:34.898959 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-0976e71b-92d5-43ee-9c98-44a3f941cadf tempest-TaggedBootDevicesTest-874492856 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-5f3eb6ba-d61b-4c3c-8018-5709f993e46d', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '5f3eb6ba-d61b-4c3c-8018-5709f993e46d', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:05:34.968351 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-0976e71b-92d5-43ee-9c98-44a3f941cadf tempest-TaggedBootDevicesTest-874492856 None] Connection info returned from driver {'name': 'volumes/volume-5f3eb6ba-d61b-4c3c-8018-5709f993e46d', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '5f3eb6ba-d61b-4c3c-8018-5709f993e46d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'd74d0576-9e50-435a-be18-c47178962ac6'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:05:35.106951 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-dd1befc7-4b1e-44fd-a183-5b3f138023c2 req-0976e71b-92d5-43ee-9c98-44a3f941cadf tempest-TaggedBootDevicesTest-874492856 None] attachment_update completed successfully. Oct 01 21:05:53.374098 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=25,cinder:UPDATE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:58.139504 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=49,cinder:UPDATE=17,cinder:INSERT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:05:58.152299 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-a9601990-1cb7-4d82-9935-bf03bc9262c4 req-d0eb93f2-4a52-43ca-b04d-10a3574bbf8b tempest-ServerRescueNegativeTestJSON-894078329 None] Terminate volume connection completed successfully. Oct 01 21:05:58.152299 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-a9601990-1cb7-4d82-9935-bf03bc9262c4 req-d0eb93f2-4a52-43ca-b04d-10a3574bbf8b tempest-ServerRescueNegativeTestJSON-894078329 None] Deleting attachment 7bfffbd6-e39a-4ef0-a6f3-7cfcc3753175. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:06:08.655774 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:06:08.719748 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Flow 'volume_create_manager' (fc078c98-e8ed-4a1f-b536-ceb5d84ac377) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:06:08.742754 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bd40e194-ff91-4f27-850a-98e12eaef817) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:08.796530 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bd40e194-ff91-4f27-850a-98e12eaef817) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:06:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServersNegativeTestJSON-volume-544525420',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b4cffd31-a74b-413e-8ab4-983abfe9b5c2,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6c0ad82595344eb4a1122d294faf15b2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:06:09Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:06:09Z,use_quota=True,user_id='6d63c633112e49d091421c84cea3422b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:08.816914 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e8822518-3652-408f-883e-91fc021bef26) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:08.829462 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e8822518-3652-408f-883e-91fc021bef26) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:08.844041 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e06da3c1-3cc6-485b-b31c-603128448b6b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:08.862310 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e06da3c1-3cc6-485b-b31c-603128448b6b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b4cffd31-a74b-413e-8ab4-983abfe9b5c2', 'volume_name': 'volume-b4cffd31-a74b-413e-8ab4-983abfe9b5c2', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:08.885160 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3d18eee2-c178-439e-a5da-d072bc9578f8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:08.893701 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3d18eee2-c178-439e-a5da-d072bc9578f8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:08.902196 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (12c103f2-b9dd-4bba-9448-fcd4399f1497) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:08.903965 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Volume b4cffd31-a74b-413e-8ab4-983abfe9b5c2: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b4cffd31-a74b-413e-8ab4-983abfe9b5c2', 'volume_size': 1} Oct 01 21:06:08.904746 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] creating volume 'volume-b4cffd31-a74b-413e-8ab4-983abfe9b5c2' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:06:08.906913 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:06:09.101786 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:06:09.102258 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:06:09.105928 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (12c103f2-b9dd-4bba-9448-fcd4399f1497) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b4cffd31-a74b-413e-8ab4-983abfe9b5c2', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:09.115590 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3c413dc4-89d7-405b-ab89-2f23dd504208) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:09.134127 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Volume volume-b4cffd31-a74b-413e-8ab4-983abfe9b5c2 (b4cffd31-a74b-413e-8ab4-983abfe9b5c2): created successfully Oct 01 21:06:09.140083 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3c413dc4-89d7-405b-ab89-2f23dd504208) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:09.155856 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Flow 'volume_create_manager' (fc078c98-e8ed-4a1f-b536-ceb5d84ac377) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:06:09.178241 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-525ad7e1-1628-4656-bf7d-6873fc70fe82 tempest-ServersNegativeTestJSON-1224281476 None] Created volume successfully. Oct 01 21:06:13.228703 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:06:13.291605 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Flow 'volume_create_manager' (8dbcd778-735c-4492-af31-43d65bae3b72) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:06:13.300826 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (54988afb-e17d-4d72-af35-db75774197dc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:13.321350 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (54988afb-e17d-4d72-af35-db75774197dc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:06:13Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServersNegativeTestJSON-volume-1239039683',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d8c8f504-1bc5-41c3-a943-9b0b432fa269,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6c0ad82595344eb4a1122d294faf15b2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:06:13Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:06:13Z,use_quota=True,user_id='6d63c633112e49d091421c84cea3422b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:13.334806 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7aa98219-3953-44c5-b877-6bf6407748ae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:13.339868 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7aa98219-3953-44c5-b877-6bf6407748ae) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:13.345722 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3d98b3fb-595e-4c97-bcb9-c713d8847a27) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:13.353380 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3d98b3fb-595e-4c97-bcb9-c713d8847a27) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd8c8f504-1bc5-41c3-a943-9b0b432fa269', 'volume_name': 'volume-d8c8f504-1bc5-41c3-a943-9b0b432fa269', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:13.360474 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (afc3abe6-8ae1-4ac2-9de0-396b47b8ce2a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:13.364606 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (afc3abe6-8ae1-4ac2-9de0-396b47b8ce2a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:13.369741 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8dd66f09-57f1-4494-9005-46166ee636f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:13.371132 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Volume d8c8f504-1bc5-41c3-a943-9b0b432fa269: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d8c8f504-1bc5-41c3-a943-9b0b432fa269', 'volume_size': 1} Oct 01 21:06:13.371669 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] creating volume 'volume-d8c8f504-1bc5-41c3-a943-9b0b432fa269' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:06:13.373183 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:06:13.530643 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:06:13.531225 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:06:13.535791 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8dd66f09-57f1-4494-9005-46166ee636f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d8c8f504-1bc5-41c3-a943-9b0b432fa269', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:13.541970 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4096d1ec-bc7f-40e9-ab74-de51db83fbf4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:06:13.574370 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Volume volume-d8c8f504-1bc5-41c3-a943-9b0b432fa269 (d8c8f504-1bc5-41c3-a943-9b0b432fa269): created successfully Oct 01 21:06:13.579746 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4096d1ec-bc7f-40e9-ab74-de51db83fbf4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:06:13.590292 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Flow 'volume_create_manager' (8dbcd778-735c-4492-af31-43d65bae3b72) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:06:13.613813 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4c88d8eb-22eb-48e7-95d5-55eef4026185 tempest-ServersNegativeTestJSON-1224281476 None] Created volume successfully. Oct 01 21:06:19.567634 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] Acquiring lock "cinder-1067c786-686b-41aa-b9ca-31ea24d10308-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:06:19.581134 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] Lock "cinder-1067c786-686b-41aa-b9ca-31ea24d10308-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:06:19.644234 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:06:19.732296 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:06:19.732823 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] Volume volume-1067c786-686b-41aa-b9ca-31ea24d10308 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:06:19.741415 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] deleting rbd volume volume-1067c786-686b-41aa-b9ca-31ea24d10308 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:06:20.117728 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] Created reservations ['e041c9f0-b27f-4a3a-8df8-b1c4143dd5ed', 'e86e1d79-c99e-477f-b297-4acc9d42868d', '4d026447-3c34-4b28-b479-03135b8781fd', '034b7f58-1c80-4100-b311-3e14d4465ea5'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:06:20.203539 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:06:20.268008 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:06:20.330144 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:06:20.354004 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] Deleted volume successfully. Oct 01 21:06:20.393500 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fb00ce83-a026-470d-81ed-343794fcf196 tempest-ServerRescueNegativeTestJSON-894078329 None] Lock "cinder-1067c786-686b-41aa-b9ca-31ea24d10308-delete_volume" released by "delete_volume" :: held 0.813s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:06:20.977630 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] Acquiring lock "cinder-d75ab76e-9835-456b-8a91-4e4c433fd49b-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:06:20.993443 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] Lock "cinder-d75ab76e-9835-456b-8a91-4e4c433fd49b-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:06:21.125494 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:06:21.233895 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:06:21.234190 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] Volume volume-d75ab76e-9835-456b-8a91-4e4c433fd49b is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:06:21.246246 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] deleting rbd volume volume-d75ab76e-9835-456b-8a91-4e4c433fd49b {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:06:21.629861 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] Created reservations ['fa3e55bc-68ad-4592-8fd0-c00fab3603cd', 'b99f256f-7aef-4f32-8b99-325ed75eb9fd', 'f00970f4-8bf2-497f-911d-facbe01c6b9a', '3fe06d21-2eda-442f-bf25-bc6be02352a7'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:06:21.702201 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:06:21.798305 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:06:21.853375 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:06:21.857985 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] Deleted volume successfully. Oct 01 21:06:21.885286 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-25a1b17d-472a-41b3-a845-42905aa82652 tempest-ServerRescueNegativeTestJSON-894078329 None] Lock "cinder-d75ab76e-9835-456b-8a91-4e4c433fd49b-delete_volume" released by "delete_volume" :: held 0.891s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:06:23.454455 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=39,cinder:UPDATE=20,cinder:DELETE=2,cinder:INSERT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:23.723639 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:06:23.725692 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:06:23.830998 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:06:23.907175 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:06:33.454471 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=38,cinder:UPDATE=21,cinder:INSERT=4,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:06:43.470421 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:03.463021 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:13.462932 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:23.466976 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:23.727255 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:07:23.727804 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:07:23.805755 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:07:23.927016 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:07:34.030611 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:07:34.082022 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Flow 'volume_create_manager' (b043e876-d7e8-4d50-89f7-50305999322d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:07:34.107332 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7b4e4df9-a0d4-40b7-939c-447e0aa5c6ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:07:34.144834 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7b4e4df9-a0d4-40b7-939c-447e0aa5c6ba) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:07:34Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-606581366',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f05a8921-7393-4c72-bce8-e68fb45486ed,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0b5bb0c08a5145369fc64e13d190c988',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:07:34Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:07:34Z,use_quota=True,user_id='f74f251052ee4c9a99db0c43bc00da9f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:07:34.161524 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (44d7015c-6fd2-416b-bbee-3862a4df08c4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:07:34.176831 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (44d7015c-6fd2-416b-bbee-3862a4df08c4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:07:34.192271 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a8f08e3d-f382-48b5-97c8-06b14793d40f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:07:34.199296 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a8f08e3d-f382-48b5-97c8-06b14793d40f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f05a8921-7393-4c72-bce8-e68fb45486ed', 'volume_name': 'volume-f05a8921-7393-4c72-bce8-e68fb45486ed', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:07:34.204734 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (441a7da9-7d12-4347-bedc-575fdece4a88) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:07:34.214284 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (441a7da9-7d12-4347-bedc-575fdece4a88) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:07:34.243955 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6f1ce0ec-d9de-498b-87db-2741a7e4209f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:07:34.246504 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Volume f05a8921-7393-4c72-bce8-e68fb45486ed: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f05a8921-7393-4c72-bce8-e68fb45486ed', 'volume_size': 1} Oct 01 21:07:34.247503 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] creating volume 'volume-f05a8921-7393-4c72-bce8-e68fb45486ed' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:07:34.248301 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:07:34.432994 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:07:34.433367 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:07:34.437613 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6f1ce0ec-d9de-498b-87db-2741a7e4209f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f05a8921-7393-4c72-bce8-e68fb45486ed', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:07:34.445939 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fa6c7cfa-75ec-4b6e-be78-5ae2a96a2495) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:07:34.469575 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Volume volume-f05a8921-7393-4c72-bce8-e68fb45486ed (f05a8921-7393-4c72-bce8-e68fb45486ed): created successfully Oct 01 21:07:34.474865 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fa6c7cfa-75ec-4b6e-be78-5ae2a96a2495) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:07:34.495020 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Flow 'volume_create_manager' (b043e876-d7e8-4d50-89f7-50305999322d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:07:34.523742 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-cb223e69-efaf-4954-b0a6-f79d5b291163 tempest-AttachVolumeTestJSON-1725633688 None] Created volume successfully. Oct 01 21:07:36.916749 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-ac8f2848-09df-42a2-8de6-afb47b1cac1b req-a23be9f2-2f59-4bc5-bc78-718355054152 tempest-AttachVolumeTestJSON-1725633688 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:07:38.417847 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-ac8f2848-09df-42a2-8de6-afb47b1cac1b req-a23be9f2-2f59-4bc5-bc78-718355054152 tempest-AttachVolumeTestJSON-1725633688 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.499s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:07:38.420503 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-ac8f2848-09df-42a2-8de6-afb47b1cac1b req-a23be9f2-2f59-4bc5-bc78-718355054152 tempest-AttachVolumeTestJSON-1725633688 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f05a8921-7393-4c72-bce8-e68fb45486ed', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'f05a8921-7393-4c72-bce8-e68fb45486ed', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:07:38.485208 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-ac8f2848-09df-42a2-8de6-afb47b1cac1b req-a23be9f2-2f59-4bc5-bc78-718355054152 tempest-AttachVolumeTestJSON-1725633688 None] Connection info returned from driver {'name': 'volumes/volume-f05a8921-7393-4c72-bce8-e68fb45486ed', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'f05a8921-7393-4c72-bce8-e68fb45486ed', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '8d6fbdb8-beb4-48bd-826e-a64643b5215f'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:07:38.598139 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-ac8f2848-09df-42a2-8de6-afb47b1cac1b req-a23be9f2-2f59-4bc5-bc78-718355054152 tempest-AttachVolumeTestJSON-1725633688 None] attachment_update completed successfully. Oct 01 21:07:53.462569 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=16,cinder:UPDATE=6 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:07:53.465151 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=23,cinder:UPDATE=3,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:13.467795 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:23.724956 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:08:23.726458 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:23.843947 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:24.003622 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:08:32.473667 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=2,cinder:SELECT=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:32.489607 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-04966ebe-38bf-4aa4-b28b-9a2a1370babd req-ae20a05c-3c9d-48a0-8bc1-6512b4e9f352 tempest-AttachVolumeTestJSON-1725633688 None] Terminate volume connection completed successfully. Oct 01 21:08:32.490760 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-04966ebe-38bf-4aa4-b28b-9a2a1370babd req-ae20a05c-3c9d-48a0-8bc1-6512b4e9f352 tempest-AttachVolumeTestJSON-1725633688 None] Deleting attachment 8d6fbdb8-beb4-48bd-826e-a64643b5215f. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:08:34.975989 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f78d65a7-e879-4dd0-8659-62729b1be3fd tempest-GroupSnapshotsTest-1179824992 None] Group tempest-GroupSnapshotsTest-Group-1229608500: creating Oct 01 21:08:35.016042 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f78d65a7-e879-4dd0-8659-62729b1be3fd tempest-GroupSnapshotsTest-1179824992 None] Group tempest-GroupSnapshotsTest-Group-1229608500: created successfully Oct 01 21:08:35.026225 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f78d65a7-e879-4dd0-8659-62729b1be3fd tempest-GroupSnapshotsTest-1179824992 None] Create group completed successfully. Oct 01 21:08:37.246104 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:08:37.295311 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Flow 'volume_create_manager' (56d3838b-7a1f-40ce-8d3d-db32badd2001) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:08:37.306717 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (154d20eb-762c-4ba6-8dba-d20a0e77e791) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:37.357498 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (154d20eb-762c-4ba6-8dba-d20a0e77e791) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:08:37Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1555444862',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=1b8f578a-ebc3-44c5-97f8-ca14886e5300,host='ncdv-master-1@ceph#ceph',id=dd52d52d-a9fa-4b2a-a604-1936d934ebd2,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2be32d5a16f7453c96de3dbdb05fe6d1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:08:37Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:08:37Z,use_quota=True,user_id='e683d459ead14ddca5acb36ae9c1cbed',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(3eab17e8-ca5b-4c07-8080-260aab23d34b),volume_type_id=3eab17e8-ca5b-4c07-8080-260aab23d34b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:37.370785 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2b1fcfb7-020e-47d5-88b0-19181e1e8908) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:37.375165 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2b1fcfb7-020e-47d5-88b0-19181e1e8908) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:37.385201 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8069d287-eea4-4be3-84c6-776ba7b4b0c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:37.395171 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8069d287-eea4-4be3-84c6-776ba7b4b0c3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'dd52d52d-a9fa-4b2a-a604-1936d934ebd2', 'volume_name': 'volume-dd52d52d-a9fa-4b2a-a604-1936d934ebd2', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:37.406328 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4ce91486-5494-4edb-9882-4874b6918816) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:37.425001 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4ce91486-5494-4edb-9882-4874b6918816) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:37.436310 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (96f22fbb-ddb7-4dab-b778-02d7d4d1eda1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:37.441478 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Volume dd52d52d-a9fa-4b2a-a604-1936d934ebd2: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-dd52d52d-a9fa-4b2a-a604-1936d934ebd2', 'volume_size': 1} Oct 01 21:08:37.442354 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] creating volume 'volume-dd52d52d-a9fa-4b2a-a604-1936d934ebd2' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:08:37.443420 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:37.594568 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:08:37.594886 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:08:37.598702 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (96f22fbb-ddb7-4dab-b778-02d7d4d1eda1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-dd52d52d-a9fa-4b2a-a604-1936d934ebd2', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:37.609481 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4166d974-be44-4891-a266-b419c296458f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:37.631982 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Volume volume-dd52d52d-a9fa-4b2a-a604-1936d934ebd2 (dd52d52d-a9fa-4b2a-a604-1936d934ebd2): created successfully Oct 01 21:08:37.635685 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4166d974-be44-4891-a266-b419c296458f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:37.645997 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Flow 'volume_create_manager' (56d3838b-7a1f-40ce-8d3d-db32badd2001) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:08:37.666666 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-068f0417-8772-4d12-a583-3e071ef0420f tempest-GroupSnapshotsTest-1179824992 None] Created volume successfully. Oct 01 21:08:38.627965 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-905c1679-d79a-479c-af92-09450102f3df tempest-GroupSnapshotsTest-1179824992 None] GroupSnapshot 0c97cf5a-4955-47e6-a784-75dcd8cde58c: creating. Oct 01 21:08:38.711256 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-905c1679-d79a-479c-af92-09450102f3df tempest-GroupSnapshotsTest-1179824992 None] Group snapshot 0c97cf5a-4955-47e6-a784-75dcd8cde58c: creating. {{(pid=133391) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Oct 01 21:08:38.747086 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-905c1679-d79a-479c-af92-09450102f3df tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:39.789500 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-905c1679-d79a-479c-af92-09450102f3df tempest-GroupSnapshotsTest-1179824992 None] group_snapshot 0c97cf5a-4955-47e6-a784-75dcd8cde58c: created successfully Oct 01 21:08:42.042439 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8d03ddfb-f56f-41b8-bea0-3baabfdaf00a tempest-GroupSnapshotsTest-1179824992 None] cloning volumes/volume-dd52d52d-a9fa-4b2a-a604-1936d934ebd2@snapshot-d9083f1e-e169-41c5-8ad4-e771379b92f6 to volume-c85cfe5b-c4ba-4178-907d-7c34e977c3ef {{(pid=133391) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Oct 01 21:08:42.045557 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8d03ddfb-f56f-41b8-bea0-3baabfdaf00a tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:42.169845 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8d03ddfb-f56f-41b8-bea0-3baabfdaf00a tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:42.418983 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8d03ddfb-f56f-41b8-bea0-3baabfdaf00a tempest-GroupSnapshotsTest-1179824992 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:08:42.419533 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8d03ddfb-f56f-41b8-bea0-3baabfdaf00a tempest-GroupSnapshotsTest-1179824992 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:08:42.435145 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8d03ddfb-f56f-41b8-bea0-3baabfdaf00a tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:42.526316 ncdv-master-1 cinder-volume[133391]: WARNING cinder.volume.drivers.rbd [None req-8d03ddfb-f56f-41b8-bea0-3baabfdaf00a tempest-GroupSnapshotsTest-1179824992 None] Not using v2 clone API, please upgrade to mimic+ and set the OSD minimum client compat version to mimic for better performance, fewer deletion issues: NotImplementedError Oct 01 21:08:42.682532 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8d03ddfb-f56f-41b8-bea0-3baabfdaf00a tempest-GroupSnapshotsTest-1179824992 None] Create group from source-snapshot-0c97cf5a-4955-47e6-a784-75dcd8cde58c completed successfully. Oct 01 21:08:44.129127 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=41,cinder:UPDATE=8 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:44.142709 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f53f6ed3-09e1-496f-bc07-bf48ea001d41 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:44.294412 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f53f6ed3-09e1-496f-bc07-bf48ea001d41 tempest-GroupSnapshotsTest-1179824992 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:08:44.307666 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f53f6ed3-09e1-496f-bc07-bf48ea001d41 tempest-GroupSnapshotsTest-1179824992 None] deleting rbd volume volume-c85cfe5b-c4ba-4178-907d-7c34e977c3ef {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:08:44.770975 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-f53f6ed3-09e1-496f-bc07-bf48ea001d41 tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['fb9c9841-e4bd-450d-bcc1-7509177aea26'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:08:44.837383 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-f53f6ed3-09e1-496f-bc07-bf48ea001d41 tempest-GroupSnapshotsTest-1179824992 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-2091610343 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-2091610343, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:08:44.841494 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-f53f6ed3-09e1-496f-bc07-bf48ea001d41 tempest-GroupSnapshotsTest-1179824992 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-2091610343 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-2091610343, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:08:44.893221 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-f53f6ed3-09e1-496f-bc07-bf48ea001d41 tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['0791c091-e6f6-463c-8fe4-d38af8047362', 'd6ef5a1d-9735-49a6-b4de-e7ec3bdcf141', '75d1df42-fbcb-4ed1-8650-a820f6268ca9', '75edced7-0bc8-4835-99f7-6055f97c9532'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:08:45.128305 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f53f6ed3-09e1-496f-bc07-bf48ea001d41 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:45.203454 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f53f6ed3-09e1-496f-bc07-bf48ea001d41 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:45.271537 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-f53f6ed3-09e1-496f-bc07-bf48ea001d41 tempest-GroupSnapshotsTest-1179824992 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:08:45.278938 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f53f6ed3-09e1-496f-bc07-bf48ea001d41 tempest-GroupSnapshotsTest-1179824992 None] Delete group completed successfully. Oct 01 21:08:45.444642 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-2058073f-b386-4c27-8987-438eed744155 tempest-GroupSnapshotsTest-1179824992 None] group_snapshot 0c97cf5a-4955-47e6-a784-75dcd8cde58c: deleting Oct 01 21:08:45.495100 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2058073f-b386-4c27-8987-438eed744155 tempest-GroupSnapshotsTest-1179824992 None] group_snapshot 0c97cf5a-4955-47e6-a784-75dcd8cde58c: deleting {{(pid=133391) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Oct 01 21:08:45.634404 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2058073f-b386-4c27-8987-438eed744155 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:46.373222 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-2058073f-b386-4c27-8987-438eed744155 tempest-GroupSnapshotsTest-1179824992 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-2091610343 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-2091610343, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:08:46.426769 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-2058073f-b386-4c27-8987-438eed744155 tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['ac42900a-4e2c-48b8-bc19-b75c1a2e518e', 'bc24ddb8-a61b-4e5b-8148-7a4497e2b30f', '4c06c011-c522-4db0-9113-dc5157b61ca3', 'c8635417-2f1c-44a7-83aa-421b46becc8b'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:08:46.633187 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-2058073f-b386-4c27-8987-438eed744155 tempest-GroupSnapshotsTest-1179824992 None] group_snapshot 0c97cf5a-4955-47e6-a784-75dcd8cde58c: deleted successfully Oct 01 21:08:47.563354 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=66,cinder:UPDATE=30,cinder:DELETE=1,cinder:INSERT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:47.578243 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a8ef62a1-7707-4303-8fe7-11a44fbfcf2b tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:47.741916 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a8ef62a1-7707-4303-8fe7-11a44fbfcf2b tempest-GroupSnapshotsTest-1179824992 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:08:47.742903 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a8ef62a1-7707-4303-8fe7-11a44fbfcf2b tempest-GroupSnapshotsTest-1179824992 None] Volume volume-dd52d52d-a9fa-4b2a-a604-1936d934ebd2 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:08:47.755166 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a8ef62a1-7707-4303-8fe7-11a44fbfcf2b tempest-GroupSnapshotsTest-1179824992 None] deleting rbd volume volume-dd52d52d-a9fa-4b2a-a604-1936d934ebd2 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:08:48.244026 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-a8ef62a1-7707-4303-8fe7-11a44fbfcf2b tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['1c6779c9-692f-45c9-9b80-9b1776182dbd'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:08:48.335187 ncdv-master-1 cinder-volume[133391]: WARNING cinder.db.sqlalchemy.api [None req-a8ef62a1-7707-4303-8fe7-11a44fbfcf2b tempest-GroupSnapshotsTest-1179824992 None] Reservation would make usage less than 0 for the following resources, so on commit they will be limited to prevent going below 0: ['gigabytes', 'gigabytes_tempest-GroupSnapshotsTest-volume-type-2091610343'] Oct 01 21:08:48.340082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-a8ef62a1-7707-4303-8fe7-11a44fbfcf2b tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['9c30e645-01a8-4cf1-9cfa-944d9ee7596a', '7b8484cf-5efe-4adc-b807-d4f78dcbde23', 'e151ae57-9bdf-43cd-9992-fedf00682a54', 'aec67907-ea05-4958-8330-4aaa4a0ea800'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:08:48.547824 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a8ef62a1-7707-4303-8fe7-11a44fbfcf2b tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:48.662247 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a8ef62a1-7707-4303-8fe7-11a44fbfcf2b tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:48.787482 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-a8ef62a1-7707-4303-8fe7-11a44fbfcf2b tempest-GroupSnapshotsTest-1179824992 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:08:48.793389 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a8ef62a1-7707-4303-8fe7-11a44fbfcf2b tempest-GroupSnapshotsTest-1179824992 None] Delete group completed successfully. Oct 01 21:08:49.507982 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e21ea247-fdd1-4a92-a7cf-d3a924ebe5af tempest-GroupSnapshotsTest-1179824992 None] Group tempest-GroupSnapshotsTest-Group-677779188: creating Oct 01 21:08:49.544761 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e21ea247-fdd1-4a92-a7cf-d3a924ebe5af tempest-GroupSnapshotsTest-1179824992 None] Group tempest-GroupSnapshotsTest-Group-677779188: created successfully Oct 01 21:08:49.567911 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e21ea247-fdd1-4a92-a7cf-d3a924ebe5af tempest-GroupSnapshotsTest-1179824992 None] Create group completed successfully. Oct 01 21:08:51.144534 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:08:51.173519 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Flow 'volume_create_manager' (f60cb21a-dba3-47f4-9743-5887a26607ab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:08:51.186723 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (da2e9820-e976-4e90-b265-89e35ed13fac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:51.208975 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (da2e9820-e976-4e90-b265-89e35ed13fac) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:08:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1387552500',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=a572fc2d-e030-4d83-807e-e846a7d1a22d,host='ncdv-master-1@ceph#ceph',id=2833ac85-6454-478c-9104-7bf1eca130d5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2be32d5a16f7453c96de3dbdb05fe6d1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:08:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:08:51Z,use_quota=True,user_id='e683d459ead14ddca5acb36ae9c1cbed',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(4690773f-f3a3-4bd5-a5e7-034be6365e74),volume_type_id=4690773f-f3a3-4bd5-a5e7-034be6365e74)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:51.217859 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d50c70de-a050-4d9d-936c-b31ed7f0b244) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:51.225285 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d50c70de-a050-4d9d-936c-b31ed7f0b244) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:51.231854 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e82bf0f6-db0f-4476-8b36-73ac6a952ec9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:51.237437 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e82bf0f6-db0f-4476-8b36-73ac6a952ec9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2833ac85-6454-478c-9104-7bf1eca130d5', 'volume_name': 'volume-2833ac85-6454-478c-9104-7bf1eca130d5', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:51.247121 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9b189c2b-bf15-4fd9-8f7d-381ada63657c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:51.252288 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9b189c2b-bf15-4fd9-8f7d-381ada63657c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:51.258038 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (909f99e6-e769-4e3a-b326-c3bf1dca0701) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:51.259965 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Volume 2833ac85-6454-478c-9104-7bf1eca130d5: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2833ac85-6454-478c-9104-7bf1eca130d5', 'volume_size': 1} Oct 01 21:08:51.260581 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] creating volume 'volume-2833ac85-6454-478c-9104-7bf1eca130d5' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:08:51.262395 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:51.378238 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:08:51.380683 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:08:51.383635 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (909f99e6-e769-4e3a-b326-c3bf1dca0701) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2833ac85-6454-478c-9104-7bf1eca130d5', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:51.391960 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8565950e-b701-4599-aeae-271c5184dd84) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:51.414448 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Volume volume-2833ac85-6454-478c-9104-7bf1eca130d5 (2833ac85-6454-478c-9104-7bf1eca130d5): created successfully Oct 01 21:08:51.421767 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8565950e-b701-4599-aeae-271c5184dd84) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:51.435532 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Flow 'volume_create_manager' (f60cb21a-dba3-47f4-9743-5887a26607ab) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:08:51.456311 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-fee961df-907c-481c-bfed-d0f7e45ddbba tempest-GroupSnapshotsTest-1179824992 None] Created volume successfully. Oct 01 21:08:53.310364 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:08:53.383727 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Flow 'volume_create_manager' (c2e866c0-a82b-4d87-a5f5-695cba516f48) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:08:53.404164 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2b7f0880-4a46-4c9c-8ba7-458ee5ae0da3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:53.443716 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2b7f0880-4a46-4c9c-8ba7-458ee5ae0da3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:08:53Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1146463946',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=a572fc2d-e030-4d83-807e-e846a7d1a22d,host='ncdv-master-1@ceph#ceph',id=e6f4079c-602c-40d0-9e38-33693402c8c0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2be32d5a16f7453c96de3dbdb05fe6d1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:08:53Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:08:53Z,use_quota=True,user_id='e683d459ead14ddca5acb36ae9c1cbed',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(4690773f-f3a3-4bd5-a5e7-034be6365e74),volume_type_id=4690773f-f3a3-4bd5-a5e7-034be6365e74)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:53.478274 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8ca74929-1064-4408-97ce-9ef742b0f26a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:53.493097 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8ca74929-1064-4408-97ce-9ef742b0f26a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:53.508628 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5305c902-afb2-4614-b961-de59257425d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:53.528557 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5305c902-afb2-4614-b961-de59257425d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e6f4079c-602c-40d0-9e38-33693402c8c0', 'volume_name': 'volume-e6f4079c-602c-40d0-9e38-33693402c8c0', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:53.542622 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (10483d8f-b8c0-461b-8fa2-0f471e3fb5e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:53.553807 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (10483d8f-b8c0-461b-8fa2-0f471e3fb5e5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:53.585951 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f0696870-17e9-4f02-95f0-8b5a84dab5b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:53.592621 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Volume e6f4079c-602c-40d0-9e38-33693402c8c0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e6f4079c-602c-40d0-9e38-33693402c8c0', 'volume_size': 1} Oct 01 21:08:53.593945 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] creating volume 'volume-e6f4079c-602c-40d0-9e38-33693402c8c0' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:08:53.594871 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:08:53.724554 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:08:53.725090 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:08:53.728937 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f0696870-17e9-4f02-95f0-8b5a84dab5b9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e6f4079c-602c-40d0-9e38-33693402c8c0', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:53.751368 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d13bcd6c-85f5-4c6e-8712-44e1195d8662) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:08:53.771649 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Volume volume-e6f4079c-602c-40d0-9e38-33693402c8c0 (e6f4079c-602c-40d0-9e38-33693402c8c0): created successfully Oct 01 21:08:53.775777 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d13bcd6c-85f5-4c6e-8712-44e1195d8662) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:08:53.803845 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Flow 'volume_create_manager' (c2e866c0-a82b-4d87-a5f5-695cba516f48) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:08:53.843380 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-51eb7c7e-eace-472d-88ae-d95450e0f82f tempest-GroupSnapshotsTest-1179824992 None] Created volume successfully. Oct 01 21:08:54.530434 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=62,cinder:INSERT=7,cinder:UPDATE=29,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:08:54.609178 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e2c00fea-c1bf-4224-9562-826f7904afb1 tempest-GroupSnapshotsTest-1179824992 None] Update group completed successfully. Oct 01 21:08:55.910444 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-af5009df-af29-4428-b250-ee5c9d6e1a47 tempest-GroupSnapshotsTest-1179824992 None] Update group completed successfully. Oct 01 21:08:57.024826 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-cb17b7c6-d697-4672-94e7-08433f336c82 tempest-GroupSnapshotsTest-1179824992 None] Update group completed successfully. Oct 01 21:08:58.220736 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-81064dbb-fbeb-42af-92f3-2f4507fda5b6 tempest-GroupSnapshotsTest-1179824992 None] Update group completed successfully. Oct 01 21:08:59.645005 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-46fe805b-2618-49fc-8298-e75159da54fe tempest-GroupSnapshotsTest-1179824992 None] GroupSnapshot e2550d35-11f0-4451-94a4-ba059228e3a3: creating. Oct 01 21:08:59.670988 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-46fe805b-2618-49fc-8298-e75159da54fe tempest-GroupSnapshotsTest-1179824992 None] Group snapshot e2550d35-11f0-4451-94a4-ba059228e3a3: creating. {{(pid=133391) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Oct 01 21:08:59.692324 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-46fe805b-2618-49fc-8298-e75159da54fe tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:00.733985 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-46fe805b-2618-49fc-8298-e75159da54fe tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:01.929749 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-46fe805b-2618-49fc-8298-e75159da54fe tempest-GroupSnapshotsTest-1179824992 None] group_snapshot e2550d35-11f0-4451-94a4-ba059228e3a3: created successfully Oct 01 21:09:03.011915 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9dba8f21-3034-4128-be83-39e2ad7dc15a tempest-GroupSnapshotsTest-1179824992 None] group_snapshot e2550d35-11f0-4451-94a4-ba059228e3a3: deleting Oct 01 21:09:03.082476 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dba8f21-3034-4128-be83-39e2ad7dc15a tempest-GroupSnapshotsTest-1179824992 None] group_snapshot e2550d35-11f0-4451-94a4-ba059228e3a3: deleting {{(pid=133391) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Oct 01 21:09:03.138058 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9dba8f21-3034-4128-be83-39e2ad7dc15a tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:03.879608 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9dba8f21-3034-4128-be83-39e2ad7dc15a tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:04.819615 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=76,cinder:UPDATE=20,cinder:INSERT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:04.879035 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-9dba8f21-3034-4128-be83-39e2ad7dc15a tempest-GroupSnapshotsTest-1179824992 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-1745966396 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-1745966396, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:09:04.879891 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-9dba8f21-3034-4128-be83-39e2ad7dc15a tempest-GroupSnapshotsTest-1179824992 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-1745966396 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-1745966396, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:09:04.896682 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-9dba8f21-3034-4128-be83-39e2ad7dc15a tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['5278a993-ed60-4793-a33f-059d4b564923', 'd2ee7c47-b9d5-468e-b4cc-65f654e4ad57', 'e80f07b6-55be-4b98-921f-4d993c41316f', 'd2019d3c-52f8-48a2-b94b-e750a6c2b1bd'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:04.996602 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-9dba8f21-3034-4128-be83-39e2ad7dc15a tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['138cd72c-39e4-45cd-bfd8-00c63169fcc9', 'd8b90e65-3eea-4975-b185-7b12cf609dd5', 'acd88eee-9d15-40f2-aa7b-eaf56d578dc8', '22fac935-b7bc-4218-bb28-f271f504f40e'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:05.054636 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9dba8f21-3034-4128-be83-39e2ad7dc15a tempest-GroupSnapshotsTest-1179824992 None] group_snapshot e2550d35-11f0-4451-94a4-ba059228e3a3: deleted successfully Oct 01 21:09:06.105950 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:06.200392 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:09:06.201030 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] Volume volume-2833ac85-6454-478c-9104-7bf1eca130d5 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:09:06.221055 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] deleting rbd volume volume-2833ac85-6454-478c-9104-7bf1eca130d5 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:09:06.494133 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:06.564215 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:09:06.564736 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] Volume volume-e6f4079c-602c-40d0-9e38-33693402c8c0 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:09:06.571804 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] deleting rbd volume volume-e6f4079c-602c-40d0-9e38-33693402c8c0 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:09:06.945619 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['973c91f9-c465-4cdf-a35c-d4dcdf8d47fe'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:06.973909 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-1745966396 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-1745966396, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:09:06.997093 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['9db99baa-680a-42b2-8e31-0adc28aba2b3', '73472817-5841-44bd-a797-ddffc305e055', '572c6913-37be-41b9-85e4-80247a852105', 'e3700b5e-7592-4b14-985c-26c11966633e'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:07.011330 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=75,cinder:UPDATE=18,cinder:INSERT=7 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:07.120172 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['71d22497-4794-47e8-9982-9996d703cd88', 'afe3146d-3d56-4c5f-b6b2-563263fa182e', '9a608261-b36b-482c-9381-bd0bfc8aa40b', 'f3666d61-8c6c-4128-a929-f7e1f32777ef'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:07.263094 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:07.340941 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:07.407463 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:09:07.423954 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3e2b8b0e-e1d3-4d66-9fb5-e3f34fc56048 tempest-GroupSnapshotsTest-1179824992 None] Delete group completed successfully. Oct 01 21:09:07.746307 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-bf1390bf-3a46-47b8-9ed9-853d5ee22b4d tempest-GroupSnapshotsTest-1179824992 None] Group tempest-GroupSnapshotsTest-Group-1988798974: creating Oct 01 21:09:07.788343 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-bf1390bf-3a46-47b8-9ed9-853d5ee22b4d tempest-GroupSnapshotsTest-1179824992 None] Group tempest-GroupSnapshotsTest-Group-1988798974: created successfully Oct 01 21:09:07.811729 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-bf1390bf-3a46-47b8-9ed9-853d5ee22b4d tempest-GroupSnapshotsTest-1179824992 None] Create group completed successfully. Oct 01 21:09:09.565195 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:09:09.615446 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Flow 'volume_create_manager' (76de7603-aeef-40de-b1cf-424bba0230c8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:09:09.637283 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (859d5381-240b-4fce-b76e-ab9387ed6caf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:09.662282 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (859d5381-240b-4fce-b76e-ab9387ed6caf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:09:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1885779171',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=fd004223-e6a8-472a-9ee6-f8a99c3b539a,host='ncdv-master-1@ceph#ceph',id=507672ef-cd95-4c50-ba6e-4b30dd4c9178,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2be32d5a16f7453c96de3dbdb05fe6d1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:09:09Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:09:09Z,use_quota=True,user_id='e683d459ead14ddca5acb36ae9c1cbed',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0afc3445-31f3-4924-afc6-79007e9d91f9),volume_type_id=0afc3445-31f3-4924-afc6-79007e9d91f9)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:09.668646 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f556fd5a-75d3-4102-9a17-5186c7a8d6ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:09.674305 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f556fd5a-75d3-4102-9a17-5186c7a8d6ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:09.680377 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a84abb1d-a3ee-4317-b2a1-95df0e31de97) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:09.685617 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a84abb1d-a3ee-4317-b2a1-95df0e31de97) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '507672ef-cd95-4c50-ba6e-4b30dd4c9178', 'volume_name': 'volume-507672ef-cd95-4c50-ba6e-4b30dd4c9178', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:09.691043 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (97a06b48-93c9-4606-86e4-2166b16d508f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:09.696241 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (97a06b48-93c9-4606-86e4-2166b16d508f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:09.702595 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9492bcea-2650-4217-b614-16600e6c6ca5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:09.703914 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Volume 507672ef-cd95-4c50-ba6e-4b30dd4c9178: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-507672ef-cd95-4c50-ba6e-4b30dd4c9178', 'volume_size': 1} Oct 01 21:09:09.705701 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] creating volume 'volume-507672ef-cd95-4c50-ba6e-4b30dd4c9178' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:09:09.706683 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:09.840087 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:09:09.840525 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:09:09.849751 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9492bcea-2650-4217-b614-16600e6c6ca5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-507672ef-cd95-4c50-ba6e-4b30dd4c9178', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:09.855158 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fd0a7fd4-efbe-459f-8023-b04caa1b0619) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:09.872478 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Volume volume-507672ef-cd95-4c50-ba6e-4b30dd4c9178 (507672ef-cd95-4c50-ba6e-4b30dd4c9178): created successfully Oct 01 21:09:09.877487 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fd0a7fd4-efbe-459f-8023-b04caa1b0619) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:09.883887 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Flow 'volume_create_manager' (76de7603-aeef-40de-b1cf-424bba0230c8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:09:09.895198 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-22312605-7a72-41c9-a834-c8adb391dd2d tempest-GroupSnapshotsTest-1179824992 None] Created volume successfully. Oct 01 21:09:10.837365 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ed00a8af-b2cb-46d3-88b8-c3adb1b58316 tempest-GroupSnapshotsTest-1179824992 None] GroupSnapshot 31ada955-2e53-465c-95d5-7b5d07f78493: creating. Oct 01 21:09:10.867614 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ed00a8af-b2cb-46d3-88b8-c3adb1b58316 tempest-GroupSnapshotsTest-1179824992 None] Group snapshot 31ada955-2e53-465c-95d5-7b5d07f78493: creating. {{(pid=133391) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Oct 01 21:09:10.901787 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ed00a8af-b2cb-46d3-88b8-c3adb1b58316 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:11.989227 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=56,cinder:INSERT=7,cinder:UPDATE=37 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:12.015087 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ed00a8af-b2cb-46d3-88b8-c3adb1b58316 tempest-GroupSnapshotsTest-1179824992 None] group_snapshot 31ada955-2e53-465c-95d5-7b5d07f78493: created successfully Oct 01 21:09:13.198023 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3a6a75f3-c38d-4d3c-8329-783cb170a17d tempest-GroupSnapshotsTest-1179824992 None] group_snapshot 31ada955-2e53-465c-95d5-7b5d07f78493: deleting Oct 01 21:09:13.234842 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3a6a75f3-c38d-4d3c-8329-783cb170a17d tempest-GroupSnapshotsTest-1179824992 None] group_snapshot 31ada955-2e53-465c-95d5-7b5d07f78493: deleting {{(pid=133391) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Oct 01 21:09:13.283632 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3a6a75f3-c38d-4d3c-8329-783cb170a17d tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:13.689562 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-3a6a75f3-c38d-4d3c-8329-783cb170a17d tempest-GroupSnapshotsTest-1179824992 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-906844327 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-906844327, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:09:13.690550 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-3a6a75f3-c38d-4d3c-8329-783cb170a17d tempest-GroupSnapshotsTest-1179824992 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-906844327 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-906844327, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:09:13.705393 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3a6a75f3-c38d-4d3c-8329-783cb170a17d tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['57c67184-53b0-4638-b093-1d8f612eb67a', 'b268f352-e338-4886-ba39-8f06b9e001f7', 'db077367-571c-4f5d-bdd6-c4d3a700aa1f', 'cdcf00fb-f813-4aef-8355-e162e02e06c9'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:13.750187 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3a6a75f3-c38d-4d3c-8329-783cb170a17d tempest-GroupSnapshotsTest-1179824992 None] group_snapshot 31ada955-2e53-465c-95d5-7b5d07f78493: deleted successfully Oct 01 21:09:14.729903 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d0dcae5e-684d-480d-b336-19422ceff237 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:14.788110 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d0dcae5e-684d-480d-b336-19422ceff237 tempest-GroupSnapshotsTest-1179824992 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:09:14.788600 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d0dcae5e-684d-480d-b336-19422ceff237 tempest-GroupSnapshotsTest-1179824992 None] Volume volume-507672ef-cd95-4c50-ba6e-4b30dd4c9178 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:09:14.795500 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d0dcae5e-684d-480d-b336-19422ceff237 tempest-GroupSnapshotsTest-1179824992 None] deleting rbd volume volume-507672ef-cd95-4c50-ba6e-4b30dd4c9178 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:09:15.084992 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-d0dcae5e-684d-480d-b336-19422ceff237 tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['a9a91d42-7a35-4797-b83d-b39358f4aa19'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:15.109410 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-d0dcae5e-684d-480d-b336-19422ceff237 tempest-GroupSnapshotsTest-1179824992 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-906844327 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-906844327, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:09:15.133303 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-d0dcae5e-684d-480d-b336-19422ceff237 tempest-GroupSnapshotsTest-1179824992 None] Created reservations ['b0dde0c2-5408-43d4-aec6-47ce1cf37c3f', '53cab273-0044-436d-ac31-48515c69a4e9', 'd4b94473-b87e-4ff5-828c-0178cb11fc71', '5ea72bb2-2567-4011-9124-639fb05201f3'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:15.137969 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=63,cinder:UPDATE=29,cinder:INSERT=7,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:15.228306 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d0dcae5e-684d-480d-b336-19422ceff237 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:15.261117 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d0dcae5e-684d-480d-b336-19422ceff237 tempest-GroupSnapshotsTest-1179824992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:15.294711 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-d0dcae5e-684d-480d-b336-19422ceff237 tempest-GroupSnapshotsTest-1179824992 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:09:15.299043 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d0dcae5e-684d-480d-b336-19422ceff237 tempest-GroupSnapshotsTest-1179824992 None] Delete group completed successfully. Oct 01 21:09:19.019450 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] Acquiring lock "cinder-d8c8f504-1bc5-41c3-a943-9b0b432fa269-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:09:19.055957 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] Lock "cinder-d8c8f504-1bc5-41c3-a943-9b0b432fa269-delete_volume" acquired by "delete_volume" :: waited 0.038s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:09:19.132352 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:19.231239 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:09:19.231817 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] Volume volume-d8c8f504-1bc5-41c3-a943-9b0b432fa269 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:09:19.240710 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] deleting rbd volume volume-d8c8f504-1bc5-41c3-a943-9b0b432fa269 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:09:19.536892 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] Created reservations ['5c028faf-4d55-4510-aace-e883caa3f614', '6fbc9bf0-5255-47da-8c97-1508b6cf2973', '86d90256-becb-490c-8d41-83b35f2d5148', '1a9e2322-0ff1-4f1d-a0e6-a5142e9cd1d3'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:19.623863 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:19.685384 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:19.729897 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:09:19.734468 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] Deleted volume successfully. Oct 01 21:09:19.750110 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3e644811-f4af-4a28-b667-79dd6bbb97b3 tempest-ServersNegativeTestJSON-1224281476 None] Lock "cinder-d8c8f504-1bc5-41c3-a943-9b0b432fa269-delete_volume" released by "delete_volume" :: held 0.695s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:09:20.266374 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] Acquiring lock "cinder-b4cffd31-a74b-413e-8ab4-983abfe9b5c2-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:09:20.281982 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] Lock "cinder-b4cffd31-a74b-413e-8ab4-983abfe9b5c2-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:09:20.350188 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:20.483841 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:09:20.484190 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] Volume volume-b4cffd31-a74b-413e-8ab4-983abfe9b5c2 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:09:20.496177 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] deleting rbd volume volume-b4cffd31-a74b-413e-8ab4-983abfe9b5c2 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:09:20.966273 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=63,cinder:UPDATE=27,cinder:INSERT=9,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:20.999390 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] Created reservations ['a29c51ab-df12-493c-b305-a63fdb6b0b09', '392b3177-875c-4b64-9dcd-aa70a5a96ca3', '363ee1d3-4087-44e7-a225-a074f75f307e', 'bc087e36-6fe4-43c6-80c7-cddf5272a7f1'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:21.121945 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:21.183087 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:21.241369 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:09:21.248475 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] Deleted volume successfully. Oct 01 21:09:21.267143 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-03b6be56-e2b6-4783-855d-d2791ca49b12 tempest-ServersNegativeTestJSON-1224281476 None] Lock "cinder-b4cffd31-a74b-413e-8ab4-983abfe9b5c2-delete_volume" released by "delete_volume" :: held 0.985s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:09:21.846692 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:09:21.882063 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Flow 'volume_create_manager' (4245a646-021f-49cd-af84-9d82af278b1d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:09:21.890167 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1697a37d-2123-4878-a928-68968818856e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:21.924241 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1697a37d-2123-4878-a928-68968818856e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:09:21Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotsActionsTest-Volume-302678468',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5e0924fd-fd5b-4172-9c96-a57bb08b4321,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8fb7c0f568dc47249086aa49600baf74',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:09:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:09:22Z,use_quota=True,user_id='7f5e1c1196ea4a86a24299749d6371da',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:21.939605 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (13508656-1c38-47ce-bd96-1da5a6c7c6e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:21.950449 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (13508656-1c38-47ce-bd96-1da5a6c7c6e5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:21.965935 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a0adaf63-6e0e-4833-9530-b8c187489c5b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:21.976824 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a0adaf63-6e0e-4833-9530-b8c187489c5b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5e0924fd-fd5b-4172-9c96-a57bb08b4321', 'volume_name': 'volume-5e0924fd-fd5b-4172-9c96-a57bb08b4321', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:21.990853 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c87ae23e-0a5a-464a-b815-bb6d88380d92) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:22.001614 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c87ae23e-0a5a-464a-b815-bb6d88380d92) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:22.013813 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ac8c7a46-bce6-48ff-bb9e-af41c67a581a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:22.017128 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Volume 5e0924fd-fd5b-4172-9c96-a57bb08b4321: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5e0924fd-fd5b-4172-9c96-a57bb08b4321', 'volume_size': 1} Oct 01 21:09:22.018559 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] creating volume 'volume-5e0924fd-fd5b-4172-9c96-a57bb08b4321' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:09:22.019389 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:22.192314 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:09:22.193636 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:09:22.197740 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ac8c7a46-bce6-48ff-bb9e-af41c67a581a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5e0924fd-fd5b-4172-9c96-a57bb08b4321', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:22.205717 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (240425c0-402b-4ca2-817c-0a5e3b4cd3c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:22.232588 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Volume volume-5e0924fd-fd5b-4172-9c96-a57bb08b4321 (5e0924fd-fd5b-4172-9c96-a57bb08b4321): created successfully Oct 01 21:09:22.237610 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (240425c0-402b-4ca2-817c-0a5e3b4cd3c7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:22.256357 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Flow 'volume_create_manager' (4245a646-021f-49cd-af84-9d82af278b1d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:09:22.274221 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7d4b697f-fa06-41ce-b90c-bcd6bca63f83 tempest-SnapshotsActionsTest-301388334 None] Created volume successfully. Oct 01 21:09:23.254995 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ebd26fbf-c363-41e9-af38-a7b047256e6e tempest-SnapshotsActionsTest-301388334 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:23.698645 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ebd26fbf-c363-41e9-af38-a7b047256e6e tempest-SnapshotsActionsTest-301388334 None] Create snapshot completed successfully Oct 01 21:09:23.726925 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:09:23.727999 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:23.793416 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:23.828453 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:09:24.900961 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7135af96-91b3-4ce3-9c64-76b5790a6a72 tempest-SnapshotsActionsTest-301388334 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:26.143123 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7135af96-91b3-4ce3-9c64-76b5790a6a72 tempest-SnapshotsActionsTest-301388334 None] Create snapshot completed successfully Oct 01 21:09:27.184575 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-85a03ca2-4e36-480e-9e21-30365e6c52fe tempest-SnapshotsActionsTest-461724976 None] Acquiring lock "cinder-49bd3a72-0bbf-4bed-9cc2-0edefc476db3-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:09:27.203062 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-85a03ca2-4e36-480e-9e21-30365e6c52fe tempest-SnapshotsActionsTest-461724976 None] Lock "cinder-49bd3a72-0bbf-4bed-9cc2-0edefc476db3-delete_snapshot" acquired by "delete_snapshot" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:09:27.205981 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-85a03ca2-4e36-480e-9e21-30365e6c52fe tempest-SnapshotsActionsTest-461724976 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:28.213188 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-85a03ca2-4e36-480e-9e21-30365e6c52fe tempest-SnapshotsActionsTest-461724976 None] Created reservations ['481ccae6-02e4-4308-80c6-b7d78243030d', 'f3a8283e-3c66-4140-b568-03c32df9e375', 'e819949c-d61e-4611-b1d9-7d2b40c03af6', 'bd6c0611-3613-4ddd-8b62-51592dd74db6'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:28.290222 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-85a03ca2-4e36-480e-9e21-30365e6c52fe tempest-SnapshotsActionsTest-461724976 None] Delete snapshot completed successfully. Oct 01 21:09:28.300456 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-85a03ca2-4e36-480e-9e21-30365e6c52fe tempest-SnapshotsActionsTest-461724976 None] Lock "cinder-49bd3a72-0bbf-4bed-9cc2-0edefc476db3-delete_snapshot" released by "delete_snapshot" :: held 1.101s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:09:28.836189 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11750ae4-cd50-45ae-8dec-b21459203091 tempest-SnapshotsActionsTest-301388334 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:29.750883 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-11750ae4-cd50-45ae-8dec-b21459203091 tempest-SnapshotsActionsTest-301388334 None] Create snapshot completed successfully Oct 01 21:09:30.100636 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-61bfde31-8ea6-49bd-bcf3-d216011d0b5d tempest-SnapshotsActionsTest-461724976 None] Acquiring lock "cinder-5ac74a4f-fd06-4ff1-950b-c2170554fdc4-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:09:30.124184 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-61bfde31-8ea6-49bd-bcf3-d216011d0b5d tempest-SnapshotsActionsTest-461724976 None] Lock "cinder-5ac74a4f-fd06-4ff1-950b-c2170554fdc4-delete_snapshot" acquired by "delete_snapshot" :: waited 0.024s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:09:30.127120 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-61bfde31-8ea6-49bd-bcf3-d216011d0b5d tempest-SnapshotsActionsTest-461724976 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:31.232134 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-61bfde31-8ea6-49bd-bcf3-d216011d0b5d tempest-SnapshotsActionsTest-461724976 None] Created reservations ['ead9542d-10a0-42e5-b8a5-a31ab24d3808', 'c0a45f5c-5976-4f3c-815c-f9a14e0ed22d', 'bd61b66a-a044-4196-b6e3-b358db6eb95c', '63388e2a-995f-45e6-97ed-a8dd38e87606'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:31.263243 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=32,cinder:SELECT=58,cinder:INSERT=9,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:31.282248 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-61bfde31-8ea6-49bd-bcf3-d216011d0b5d tempest-SnapshotsActionsTest-461724976 None] Delete snapshot completed successfully. Oct 01 21:09:31.287436 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-61bfde31-8ea6-49bd-bcf3-d216011d0b5d tempest-SnapshotsActionsTest-461724976 None] Lock "cinder-5ac74a4f-fd06-4ff1-950b-c2170554fdc4-delete_snapshot" released by "delete_snapshot" :: held 1.163s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:09:32.739562 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9c5f26c3-04ff-4e3f-9e43-249f52dd5ea1 tempest-SnapshotsActionsTest-301388334 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:33.265995 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9c5f26c3-04ff-4e3f-9e43-249f52dd5ea1 tempest-SnapshotsActionsTest-301388334 None] Create snapshot completed successfully Oct 01 21:09:33.979310 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-045ad77c-8ea8-4c1c-b422-2ecf4fceb29f tempest-SnapshotsActionsTest-461724976 None] Acquiring lock "cinder-694d2414-9dca-4ca6-af5d-7b0b389f3227-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:09:33.992152 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-045ad77c-8ea8-4c1c-b422-2ecf4fceb29f tempest-SnapshotsActionsTest-461724976 None] Lock "cinder-694d2414-9dca-4ca6-af5d-7b0b389f3227-delete_snapshot" acquired by "delete_snapshot" :: waited 0.012s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:09:33.996272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-045ad77c-8ea8-4c1c-b422-2ecf4fceb29f tempest-SnapshotsActionsTest-461724976 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:34.824422 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-045ad77c-8ea8-4c1c-b422-2ecf4fceb29f tempest-SnapshotsActionsTest-461724976 None] Created reservations ['8d2c3f08-9419-4ac9-b26c-5c133d9eb30f', '13bc75de-50c4-42cc-afb2-e795477bb2ff', 'f2e1b2ea-4fbf-498d-a929-9d71ecbb7d72', '6683f788-2759-495f-b01b-fed60241add0'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:34.887127 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=33,cinder:SELECT=58,cinder:DELETE=4,cinder:INSERT=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:34.912741 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-045ad77c-8ea8-4c1c-b422-2ecf4fceb29f tempest-SnapshotsActionsTest-461724976 None] Delete snapshot completed successfully. Oct 01 21:09:34.921706 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-045ad77c-8ea8-4c1c-b422-2ecf4fceb29f tempest-SnapshotsActionsTest-461724976 None] Lock "cinder-694d2414-9dca-4ca6-af5d-7b0b389f3227-delete_snapshot" released by "delete_snapshot" :: held 0.931s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:09:35.533130 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-571931e3-a637-406a-a38a-9b478b049e1c tempest-SnapshotsActionsTest-301388334 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:36.336865 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-571931e3-a637-406a-a38a-9b478b049e1c tempest-SnapshotsActionsTest-301388334 None] Create snapshot completed successfully Oct 01 21:09:36.800123 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-13e58780-0dfd-43aa-bcce-7aa0deddd768 tempest-SnapshotsActionsTest-461724976 None] Acquiring lock "cinder-110e1c4f-d667-471c-813d-ee28c34d344b-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:09:36.817189 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-13e58780-0dfd-43aa-bcce-7aa0deddd768 tempest-SnapshotsActionsTest-461724976 None] Lock "cinder-110e1c4f-d667-471c-813d-ee28c34d344b-delete_snapshot" acquired by "delete_snapshot" :: waited 0.016s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:09:36.819155 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-13e58780-0dfd-43aa-bcce-7aa0deddd768 tempest-SnapshotsActionsTest-461724976 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:37.858295 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-13e58780-0dfd-43aa-bcce-7aa0deddd768 tempest-SnapshotsActionsTest-461724976 None] Created reservations ['848629bf-ca66-41ef-bbd8-a73eeeddcd2c', 'e411d72d-4ac1-40e2-938c-eb6e81d2ef0c', 'e37f52a5-771c-48ff-8cf5-3e5658f3cf1f', '369cc1ea-c976-489b-aecf-6db26268e123'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:37.936250 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-13e58780-0dfd-43aa-bcce-7aa0deddd768 tempest-SnapshotsActionsTest-461724976 None] Delete snapshot completed successfully. Oct 01 21:09:37.943362 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-13e58780-0dfd-43aa-bcce-7aa0deddd768 tempest-SnapshotsActionsTest-461724976 None] Lock "cinder-110e1c4f-d667-471c-813d-ee28c34d344b-delete_snapshot" released by "delete_snapshot" :: held 1.127s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:09:39.647491 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9b6c3000-bea2-4e20-be56-c043c3a55baa tempest-SnapshotsActionsTest-301388334 None] Acquiring lock "cinder-8d871868-859c-4e93-b5cf-63f1d85422c6-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:09:39.654972 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9b6c3000-bea2-4e20-be56-c043c3a55baa tempest-SnapshotsActionsTest-301388334 None] Lock "cinder-8d871868-859c-4e93-b5cf-63f1d85422c6-delete_snapshot" acquired by "delete_snapshot" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:09:39.658723 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9b6c3000-bea2-4e20-be56-c043c3a55baa tempest-SnapshotsActionsTest-301388334 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:40.741302 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-9b6c3000-bea2-4e20-be56-c043c3a55baa tempest-SnapshotsActionsTest-301388334 None] Created reservations ['5ba8fcd6-625c-4976-9e0f-0310146f655f', 'e4941491-efb0-4f4d-8fb1-b437a371bea8', 'b3fa95b7-1c72-4ae0-bf28-11a552b02e81', 'e94e6db6-643b-426c-a694-887a88e2db04'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:40.795712 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9b6c3000-bea2-4e20-be56-c043c3a55baa tempest-SnapshotsActionsTest-301388334 None] Delete snapshot completed successfully. Oct 01 21:09:40.802809 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9b6c3000-bea2-4e20-be56-c043c3a55baa tempest-SnapshotsActionsTest-301388334 None] Lock "cinder-8d871868-859c-4e93-b5cf-63f1d85422c6-delete_snapshot" released by "delete_snapshot" :: held 1.148s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:09:40.841783 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] Acquiring lock "cinder-5e0924fd-fd5b-4172-9c96-a57bb08b4321-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:09:40.856555 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] Lock "cinder-5e0924fd-fd5b-4172-9c96-a57bb08b4321-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:09:40.907510 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:40.971459 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:09:40.972051 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] Volume volume-5e0924fd-fd5b-4172-9c96-a57bb08b4321 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:09:40.978624 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] deleting rbd volume volume-5e0924fd-fd5b-4172-9c96-a57bb08b4321 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:09:41.220545 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] Created reservations ['011930c9-f988-4fee-b949-c2b011ae0cb8', 'c8364b26-5222-4f23-a24d-4510c0d6fc4e', 'ec121d50-1ae3-4361-9c9f-375bf71eccef', 'e6ad224d-a833-482e-ace4-23384eb4d919'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:09:41.297096 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=53,cinder:UPDATE=35,cinder:DELETE=2,cinder:INSERT=10 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:09:41.303363 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:41.368814 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:41.424347 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:09:41.440569 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] Deleted volume successfully. Oct 01 21:09:41.472318 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d5754665-ceed-461b-9d52-8b44ed36e4e2 tempest-SnapshotsActionsTest-301388334 None] Lock "cinder-5e0924fd-fd5b-4172-9c96-a57bb08b4321-delete_volume" released by "delete_volume" :: held 0.615s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:09:45.141218 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:09:45.171998 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Flow 'volume_create_manager' (ee87d2fe-9b39-4b78-bce9-e47fbc2eed60) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:09:45.181464 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e1dfcd7d-a957-41c0-80a6-f1529fe0edc8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:45.206223 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e1dfcd7d-a957-41c0-80a6-f1529fe0edc8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:09:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-678568809',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=45c8ab81-bd0c-4e31-bf20-a283c57ecdab,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fa37082733634ec688c9d6272a0d9b1d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:09:45Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:09:45Z,use_quota=True,user_id='8d79dff848bf40fc89377e13736c527b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:45.213271 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (00de5225-470e-4f59-9e93-af87d48b8103) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:45.219328 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (00de5225-470e-4f59-9e93-af87d48b8103) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:45.227795 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (16aa4ebe-c491-4ef4-a87f-b27c88e8bdf9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:45.232512 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (16aa4ebe-c491-4ef4-a87f-b27c88e8bdf9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '45c8ab81-bd0c-4e31-bf20-a283c57ecdab', 'volume_name': 'volume-45c8ab81-bd0c-4e31-bf20-a283c57ecdab', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:45.239413 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6c9ea0bb-b5a0-4497-bad6-3bbf03897380) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:45.245068 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6c9ea0bb-b5a0-4497-bad6-3bbf03897380) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:45.251365 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8f7e7f23-9ad8-4582-a87f-0010f8479a1c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:45.252756 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Volume 45c8ab81-bd0c-4e31-bf20-a283c57ecdab: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-45c8ab81-bd0c-4e31-bf20-a283c57ecdab', 'volume_size': 1} Oct 01 21:09:45.254459 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] creating volume 'volume-45c8ab81-bd0c-4e31-bf20-a283c57ecdab' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:09:45.255571 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:09:45.394200 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:09:45.395768 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:09:45.402575 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8f7e7f23-9ad8-4582-a87f-0010f8479a1c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-45c8ab81-bd0c-4e31-bf20-a283c57ecdab', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:45.422145 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (54f54db2-d5fd-4f6e-beb4-9ba613694f15) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:09:45.435719 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Volume volume-45c8ab81-bd0c-4e31-bf20-a283c57ecdab (45c8ab81-bd0c-4e31-bf20-a283c57ecdab): created successfully Oct 01 21:09:45.439397 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (54f54db2-d5fd-4f6e-beb4-9ba613694f15) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:09:45.460802 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Flow 'volume_create_manager' (ee87d2fe-9b39-4b78-bce9-e47fbc2eed60) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:09:45.480448 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-581ee392-d764-49f9-a695-e25553343da6 tempest-AttachVolumeShelveTestJSON-705121323 None] Created volume successfully. Oct 01 21:09:55.489015 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:03.396997 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:10:03.425705 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Flow 'volume_create_manager' (108b1202-eb72-4d70-aaef-fef98702ce89) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:03.444882 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (507f4b1e-8fbd-4135-9004-2bdddc5bdf26) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:03.485055 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (507f4b1e-8fbd-4135-9004-2bdddc5bdf26) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:10:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasAdminTestJSON-Volume-812018234',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2593c373-8642-42f8-9fdb-58cd6616ca51,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='438396a805cb4950ba2f788942425ee3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:10:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:10:03Z,use_quota=True,user_id='50dae5a392834816b8f8bf29c1756438',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:03.498891 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6842654d-0b81-40de-951f-0eb5039f9fc2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:03.508404 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6842654d-0b81-40de-951f-0eb5039f9fc2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:03.530868 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a0146230-8a79-4dc6-a049-7ef22fc7d789) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:03.539004 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a0146230-8a79-4dc6-a049-7ef22fc7d789) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2593c373-8642-42f8-9fdb-58cd6616ca51', 'volume_name': 'volume-2593c373-8642-42f8-9fdb-58cd6616ca51', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:03.548189 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8dd91f29-6a67-4db8-8c24-d2e22446b30f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:03.556447 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8dd91f29-6a67-4db8-8c24-d2e22446b30f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:03.593378 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8c2be0e0-d903-46ad-bbf2-6f6170d99adf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:03.602026 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Volume 2593c373-8642-42f8-9fdb-58cd6616ca51: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2593c373-8642-42f8-9fdb-58cd6616ca51', 'volume_size': 1} Oct 01 21:10:03.602734 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] creating volume 'volume-2593c373-8642-42f8-9fdb-58cd6616ca51' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:10:03.603562 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:03.780750 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:10:03.781373 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:10:03.790569 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8c2be0e0-d903-46ad-bbf2-6f6170d99adf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2593c373-8642-42f8-9fdb-58cd6616ca51', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:03.801999 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (531c19d2-0bdf-4c52-a579-b9286f5755bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:03.819326 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Volume volume-2593c373-8642-42f8-9fdb-58cd6616ca51 (2593c373-8642-42f8-9fdb-58cd6616ca51): created successfully Oct 01 21:10:03.827099 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (531c19d2-0bdf-4c52-a579-b9286f5755bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:03.851480 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Flow 'volume_create_manager' (108b1202-eb72-4d70-aaef-fef98702ce89) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:03.880198 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9776748b-f831-44fc-bf2f-d9cbb139b5aa tempest-VolumeQuotasAdminTestJSON-409798288 None] Created volume successfully. Oct 01 21:10:04.691828 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] Acquiring lock "cinder-2593c373-8642-42f8-9fdb-58cd6616ca51-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:10:04.772522 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] Lock "cinder-2593c373-8642-42f8-9fdb-58cd6616ca51-delete_volume" acquired by "delete_volume" :: waited 0.081s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:10:04.926856 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=46,cinder:UPDATE=21,cinder:INSERT=4,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:04.946755 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:05.070527 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:10:05.070956 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] Volume volume-2593c373-8642-42f8-9fdb-58cd6616ca51 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:10:05.083247 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] deleting rbd volume volume-2593c373-8642-42f8-9fdb-58cd6616ca51 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:10:05.525468 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] Created reservations ['2e3b98ba-ac60-4690-b1c0-107b1b2e506e', '060cb783-2aeb-4800-966c-1868fa00e0f4', '5a69831f-89a3-4583-bf53-83dbdc0f8635', '9b73a47a-54e4-4330-b83e-9ed8ed06f157'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:10:05.675988 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:05.843632 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:05.957818 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:10:05.966101 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] Deleted volume successfully. Oct 01 21:10:05.993330 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-2e67d6a5-9dc3-4a83-9942-5d952a464c79 tempest-VolumeQuotasAdminTestJSON-409798288 None] Lock "cinder-2593c373-8642-42f8-9fdb-58cd6616ca51-delete_volume" released by "delete_volume" :: held 1.221s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:10:06.889396 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:10:06.943062 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Flow 'volume_create_manager' (12b5bf25-05a6-4d21-9797-e5305a9afcdd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:06.961758 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4760112b-1994-42d5-8646-e772cb6f14a2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:07.033240 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4760112b-1994-42d5-8646-e772cb6f14a2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:10:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasAdminTestJSON-Volume-1355014131',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=324f9e0c-fab3-4a60-a66c-c959ee22fcfa,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='438396a805cb4950ba2f788942425ee3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:10:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:10:07Z,use_quota=True,user_id='50dae5a392834816b8f8bf29c1756438',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:07.039585 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (78d6c3b9-5690-4b09-bc54-2b4f731637d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:07.051797 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (78d6c3b9-5690-4b09-bc54-2b4f731637d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:07.060536 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0f834e96-7432-4320-9fbc-e369dc253f98) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:07.067787 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0f834e96-7432-4320-9fbc-e369dc253f98) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '324f9e0c-fab3-4a60-a66c-c959ee22fcfa', 'volume_name': 'volume-324f9e0c-fab3-4a60-a66c-c959ee22fcfa', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:07.087697 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7b4efe7e-fef7-49c4-a68f-c1fdd555ba29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:07.091650 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7b4efe7e-fef7-49c4-a68f-c1fdd555ba29) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:07.097057 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f792bfbb-2ed7-4724-bb51-428fc77c5495) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:07.098603 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Volume 324f9e0c-fab3-4a60-a66c-c959ee22fcfa: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-324f9e0c-fab3-4a60-a66c-c959ee22fcfa', 'volume_size': 1} Oct 01 21:10:07.099158 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] creating volume 'volume-324f9e0c-fab3-4a60-a66c-c959ee22fcfa' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:10:07.100086 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:07.261731 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:10:07.262106 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:10:07.265969 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f792bfbb-2ed7-4724-bb51-428fc77c5495) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-324f9e0c-fab3-4a60-a66c-c959ee22fcfa', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:07.279467 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (52b8c72e-aea9-41fe-b5cc-eeb06bdc7a96) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:07.306092 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Volume volume-324f9e0c-fab3-4a60-a66c-c959ee22fcfa (324f9e0c-fab3-4a60-a66c-c959ee22fcfa): created successfully Oct 01 21:10:07.309774 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (52b8c72e-aea9-41fe-b5cc-eeb06bdc7a96) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:07.320832 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Flow 'volume_create_manager' (12b5bf25-05a6-4d21-9797-e5305a9afcdd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:07.347731 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5482f7de-e123-4057-83c1-bfa427b095d9 tempest-VolumeQuotasAdminTestJSON-409798288 None] Created volume successfully. Oct 01 21:10:09.066998 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-47179956-d7b3-4775-8844-6892d0cafdf2 tempest-VolumeQuotasAdminTestJSON-1357770586 None] Transfer volume completed successfully. Oct 01 21:10:09.774994 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] Acquiring lock "cinder-324f9e0c-fab3-4a60-a66c-c959ee22fcfa-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:10:09.808383 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] Lock "cinder-324f9e0c-fab3-4a60-a66c-c959ee22fcfa-delete_volume" acquired by "delete_volume" :: waited 0.034s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:10:09.906768 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:10.054201 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:10:10.054814 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] Volume volume-324f9e0c-fab3-4a60-a66c-c959ee22fcfa is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:10:10.071249 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] deleting rbd volume volume-324f9e0c-fab3-4a60-a66c-c959ee22fcfa {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:10:10.492688 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] Created reservations ['38b33248-4663-467c-8229-4c84070b3174', '357abe6b-fb09-4ccb-815a-3ceafbde447c', '91dea36f-8622-411a-b207-48d1c83f53de', 'be42a619-75a2-499a-9065-fd32dba0679d'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:10:10.625613 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:10.731787 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:10.809790 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:10:10.815648 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] Deleted volume successfully. Oct 01 21:10:10.839088 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c37b4732-f3d2-46ab-868f-acbc64e06ea1 tempest-VolumeQuotasAdminTestJSON-301793498 None] Lock "cinder-324f9e0c-fab3-4a60-a66c-c959ee22fcfa-delete_volume" released by "delete_volume" :: held 1.031s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:10:11.105941 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:10:11.135225 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Flow 'volume_create_manager' (e0c41419-a169-4c53-8a24-6a0b847f5c63) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:11.143423 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (666b402a-3843-4bd6-89d1-306ca39e4df2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:11.165118 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (666b402a-3843-4bd6-89d1-306ca39e4df2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:10:11Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1908533613',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=057ca534-4250-412a-aa43-66638a05be1b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='1215f2e4b926472794f0e60cefa2d90e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:10:11Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:10:11Z,use_quota=True,user_id='4f3c9d35f6b843ef905f43d758a9bb25',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:11.172128 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2735bf00-d448-4a24-8832-3d34c617f9d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:11.179002 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2735bf00-d448-4a24-8832-3d34c617f9d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:11.187398 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1be243e5-36a9-4215-ab6b-f7842f6db95c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:11.385220 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1be243e5-36a9-4215-ab6b-f7842f6db95c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '057ca534-4250-412a-aa43-66638a05be1b', 'volume_name': 'volume-057ca534-4250-412a-aa43-66638a05be1b', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:11.401858 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5f51dd92-d0d8-4ce2-8b44-30815bcea332) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:11.417206 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5f51dd92-d0d8-4ce2-8b44-30815bcea332) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:11.433594 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (85216ca7-dd0d-407f-a947-1a8f332d1737) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:11.434703 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Volume 057ca534-4250-412a-aa43-66638a05be1b: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-057ca534-4250-412a-aa43-66638a05be1b', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:10:11.435283 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Cloning 057ca534-4250-412a-aa43-66638a05be1b from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:10:11.436043 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:10:11.458948 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.023s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:10:11.514664 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 10, 11, 494878)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:10:11.515432 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:10:11.516267 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:10:11.528848 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.070s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:10:11.531222 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:10:11.552800 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 10, 11, 541437)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:10:11.553931 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:10:11.555029 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:10:11.584191 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:11.709788 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] creating snapshot='volume-057ca534-4250-412a-aa43-66638a05be1b.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:10:12.137025 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-057ca534-4250-412a-aa43-66638a05be1b.clone_snap' to 'volume-057ca534-4250-412a-aa43-66638a05be1b' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:10:12.400029 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:10:12.407631 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:10:12.408100 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:10:12.448516 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:10:12.449387 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Marking volume 057ca534-4250-412a-aa43-66638a05be1b as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:10:12.474794 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to 057ca534-4250-412a-aa43-66638a05be1b. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:10:12.475310 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Creating volume glance metadata for volume 057ca534-4250-412a-aa43-66638a05be1b backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:10:12.608120 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (85216ca7-dd0d-407f-a947-1a8f332d1737) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-057ca534-4250-412a-aa43-66638a05be1b', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:12.614069 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (adf9bce4-b94a-4e4c-9358-829799a39cb8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:12.636079 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Volume volume-057ca534-4250-412a-aa43-66638a05be1b (057ca534-4250-412a-aa43-66638a05be1b): created successfully Oct 01 21:10:12.640738 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (adf9bce4-b94a-4e4c-9358-829799a39cb8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:12.665495 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Flow 'volume_create_manager' (e0c41419-a169-4c53-8a24-6a0b847f5c63) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:12.686969 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-53da3156-b428-4f2e-8ebb-7b68b6792c05 tempest-ServersTestBootFromVolume-1669384068 None] Created volume successfully. Oct 01 21:10:19.374080 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-168f4f2d-ca62-4248-98e7-810345a44ce5 req-83dee362-799d-4a8a-86bd-c51786b5bf98 tempest-ServersTestBootFromVolume-1669384068 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:10:21.241864 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-168f4f2d-ca62-4248-98e7-810345a44ce5 req-83dee362-799d-4a8a-86bd-c51786b5bf98 tempest-ServersTestBootFromVolume-1669384068 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.867s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:10:21.245998 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-168f4f2d-ca62-4248-98e7-810345a44ce5 req-83dee362-799d-4a8a-86bd-c51786b5bf98 tempest-ServersTestBootFromVolume-1669384068 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-057ca534-4250-412a-aa43-66638a05be1b', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '057ca534-4250-412a-aa43-66638a05be1b', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:10:21.289127 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-168f4f2d-ca62-4248-98e7-810345a44ce5 req-83dee362-799d-4a8a-86bd-c51786b5bf98 tempest-ServersTestBootFromVolume-1669384068 None] Connection info returned from driver {'name': 'volumes/volume-057ca534-4250-412a-aa43-66638a05be1b', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '057ca534-4250-412a-aa43-66638a05be1b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '4846d155-cf33-4dd7-9129-cc3721415614'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:10:21.392144 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-168f4f2d-ca62-4248-98e7-810345a44ce5 req-83dee362-799d-4a8a-86bd-c51786b5bf98 tempest-ServersTestBootFromVolume-1669384068 None] attachment_update completed successfully. Oct 01 21:10:23.725008 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=58,cinder:UPDATE=28,cinder:DELETE=5,cinder:INSERT=9 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:23.731138 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:10:23.734867 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:23.815587 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:23.918940 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:10:27.108345 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:10:27.172936 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Flow 'volume_create_manager' (dba31cce-53e3-4bc9-8c33-9ed04317b53d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:27.183464 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4d74087a-4a2a-44ea-979e-0c0977b539d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:27.206937 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4d74087a-4a2a-44ea-979e-0c0977b539d7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:10:27Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersTest-Volume-659142654',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='bad8134dd6f842239d31f35a9673a79a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:10:27Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:10:27Z,use_quota=True,user_id='39038d5fa6a64e3da5f4e1a135d87c2c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:27.212652 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (33403c41-e238-45ba-becb-fd1a9a44d238) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:27.217265 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (33403c41-e238-45ba-becb-fd1a9a44d238) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:27.223309 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6013ceef-2d5d-4240-bc46-806a4c748577) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:27.228534 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6013ceef-2d5d-4240-bc46-806a4c748577) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e', 'volume_name': 'volume-4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:27.235652 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (014652c3-576e-4a71-bca6-b9bb3b43fd26) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:27.240462 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (014652c3-576e-4a71-bca6-b9bb3b43fd26) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:27.245187 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (74767064-04d8-45db-a1b6-c347d3bd2aec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:27.246972 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Volume 4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e', 'volume_size': 1} Oct 01 21:10:27.247377 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] creating volume 'volume-4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:10:27.248175 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:27.362788 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:10:27.363840 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:10:27.368262 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (74767064-04d8-45db-a1b6-c347d3bd2aec) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:27.375169 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1cff7e34-80df-4e42-a262-bca131e85cf6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:27.395113 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Volume volume-4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e (4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e): created successfully Oct 01 21:10:27.403735 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1cff7e34-80df-4e42-a262-bca131e85cf6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:27.412900 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Flow 'volume_create_manager' (dba31cce-53e3-4bc9-8c33-9ed04317b53d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:27.427484 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c18ab7be-1547-47ab-a953-82ef18d1384d tempest-VolumesTransfersTest-2039664490 None] Created volume successfully. Oct 01 21:10:29.132049 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7426aa3e-2e47-40dc-afe2-c7a0b69c3961 tempest-VolumesTransfersTest-973862038 None] Transfer volume completed successfully. Oct 01 21:10:29.649600 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] Acquiring lock "cinder-4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:10:29.659364 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] Lock "cinder-4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:10:29.664243 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=66,cinder:INSERT=11,cinder:UPDATE=23 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:29.694740 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:29.763423 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:10:29.764131 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] Volume volume-4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:10:29.782454 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] deleting rbd volume volume-4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:10:30.055234 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] Created reservations ['ee35517a-6da4-4caf-85af-8889703bafad', '062df152-e62d-4b36-8793-f3afd9eb8903', '7f44c941-61c2-4d0b-8729-46cb57c9567e', '831d2276-250a-48da-8004-fd1d1ab3c4df'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:10:30.123406 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:30.187017 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:30.231307 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:10:30.240323 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] Deleted volume successfully. Oct 01 21:10:30.262256 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-74714178-3c7a-4a72-b11a-50a8928de01d tempest-VolumesTransfersTest-1122669520 None] Lock "cinder-4eaa9f0f-eacf-4dbb-a24a-a2750b57f50e-delete_volume" released by "delete_volume" :: held 0.601s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:10:31.489439 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:10:31.520698 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Flow 'volume_create_manager' (06aa8661-9b6d-448d-bb6f-663997673eee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:31.528137 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f53596cd-e54a-4c85-a27b-7244d2267a0d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:31.563163 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f53596cd-e54a-4c85-a27b-7244d2267a0d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:10:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersTest-Volume-1002683503',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8e7d8336-c063-43ac-bf9c-d2a27cde5572,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='bad8134dd6f842239d31f35a9673a79a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:10:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:10:31Z,use_quota=True,user_id='39038d5fa6a64e3da5f4e1a135d87c2c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:31.570116 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (93d7783b-c0fd-477f-b3c5-acd18a5afcdb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:31.574239 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (93d7783b-c0fd-477f-b3c5-acd18a5afcdb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:31.581274 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (31ba2dc0-a54c-473a-af07-d4a23d9fa977) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:31.586071 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (31ba2dc0-a54c-473a-af07-d4a23d9fa977) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8e7d8336-c063-43ac-bf9c-d2a27cde5572', 'volume_name': 'volume-8e7d8336-c063-43ac-bf9c-d2a27cde5572', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:31.592941 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a3389bf4-4e49-49ad-98e1-910f7cd3ff53) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:31.597085 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a3389bf4-4e49-49ad-98e1-910f7cd3ff53) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:31.602027 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (175a5d2b-8f93-455c-bf3e-b14a1130d0a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:31.603652 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Volume 8e7d8336-c063-43ac-bf9c-d2a27cde5572: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8e7d8336-c063-43ac-bf9c-d2a27cde5572', 'volume_size': 1} Oct 01 21:10:31.604204 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] creating volume 'volume-8e7d8336-c063-43ac-bf9c-d2a27cde5572' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:10:31.605141 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:31.719743 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:10:31.720149 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:10:31.724597 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (175a5d2b-8f93-455c-bf3e-b14a1130d0a3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8e7d8336-c063-43ac-bf9c-d2a27cde5572', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:31.734477 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5eb4ca9b-b2c0-44d1-b890-dfb69a8d1e52) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:31.753184 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Volume volume-8e7d8336-c063-43ac-bf9c-d2a27cde5572 (8e7d8336-c063-43ac-bf9c-d2a27cde5572): created successfully Oct 01 21:10:31.762330 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5eb4ca9b-b2c0-44d1-b890-dfb69a8d1e52) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:31.793120 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Flow 'volume_create_manager' (06aa8661-9b6d-448d-bb6f-663997673eee) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:31.817042 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-2abc6e94-96b4-4e35-9d4d-0ae294ac5838 tempest-VolumesTransfersTest-2039664490 None] Created volume successfully. Oct 01 21:10:32.446667 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-43fa0959-20e5-4833-b922-2814b40ed169 req-d33240f2-e69e-4f28-be8b-93e99d8c5153 tempest-TaggedBootDevicesTest-874492856 None] Terminate volume connection completed successfully. Oct 01 21:10:32.447361 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-43fa0959-20e5-4833-b922-2814b40ed169 req-d33240f2-e69e-4f28-be8b-93e99d8c5153 tempest-TaggedBootDevicesTest-874492856 None] Deleting attachment 67b8a6b4-6deb-4978-8d60-6840a0a81807. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:10:32.624915 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-43fa0959-20e5-4833-b922-2814b40ed169 req-eaac43d1-eb9a-47d5-8e95-46244c2da0bd tempest-TaggedBootDevicesTest-874492856 None] Terminate volume connection completed successfully. Oct 01 21:10:32.628630 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-43fa0959-20e5-4833-b922-2814b40ed169 req-eaac43d1-eb9a-47d5-8e95-46244c2da0bd tempest-TaggedBootDevicesTest-874492856 None] Deleting attachment f383a399-c27d-4f0d-9959-c87686a9c836. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:10:32.814888 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-43fa0959-20e5-4833-b922-2814b40ed169 req-220068e0-1701-4969-8b53-5d2e62af2081 tempest-TaggedBootDevicesTest-874492856 None] Terminate volume connection completed successfully. Oct 01 21:10:32.815497 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-43fa0959-20e5-4833-b922-2814b40ed169 req-220068e0-1701-4969-8b53-5d2e62af2081 tempest-TaggedBootDevicesTest-874492856 None] Deleting attachment d74d0576-9e50-435a-be18-c47178962ac6. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:10:33.061480 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] Acquiring lock "cinder-8e7d8336-c063-43ac-bf9c-d2a27cde5572-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:10:33.085870 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] Lock "cinder-8e7d8336-c063-43ac-bf9c-d2a27cde5572-delete_volume" acquired by "delete_volume" :: waited 0.024s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:10:33.135781 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:33.231438 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:10:33.232340 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] Volume volume-8e7d8336-c063-43ac-bf9c-d2a27cde5572 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:10:33.240671 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] deleting rbd volume volume-8e7d8336-c063-43ac-bf9c-d2a27cde5572 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:10:33.456465 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] Created reservations ['aaae128e-7909-4380-99ac-f26d2bb8dcfa', '10653cf2-2a5f-4df1-992e-4b07e8633e96', '2a8ccf7e-c9b1-4434-8a41-05d6e9f980d8', '82a93ae2-3540-47f2-b95b-5d49c8928b32'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:10:33.516965 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:33.607133 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:33.696816 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:10:33.713321 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] Deleted volume successfully. Oct 01 21:10:33.747802 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-64a19ff1-4c5b-44d2-8d25-da4a98209e83 tempest-VolumesTransfersTest-1122669520 None] Lock "cinder-8e7d8336-c063-43ac-bf9c-d2a27cde5572-delete_volume" released by "delete_volume" :: held 0.662s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:10:34.157288 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-1bfa66f6-f226-4ef3-a2aa-c7012f387888 req-cf82d66d-1311-4665-90db-6fce39747ae2 tempest-AttachVolumeShelveTestJSON-705121323 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:10:35.259877 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-1bfa66f6-f226-4ef3-a2aa-c7012f387888 req-cf82d66d-1311-4665-90db-6fce39747ae2 tempest-AttachVolumeShelveTestJSON-705121323 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.102s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:10:35.261112 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-1bfa66f6-f226-4ef3-a2aa-c7012f387888 req-cf82d66d-1311-4665-90db-6fce39747ae2 tempest-AttachVolumeShelveTestJSON-705121323 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-45c8ab81-bd0c-4e31-bf20-a283c57ecdab', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '45c8ab81-bd0c-4e31-bf20-a283c57ecdab', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:10:35.288793 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-1bfa66f6-f226-4ef3-a2aa-c7012f387888 req-cf82d66d-1311-4665-90db-6fce39747ae2 tempest-AttachVolumeShelveTestJSON-705121323 None] Connection info returned from driver {'name': 'volumes/volume-45c8ab81-bd0c-4e31-bf20-a283c57ecdab', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '45c8ab81-bd0c-4e31-bf20-a283c57ecdab', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '05767b8f-b005-4c0c-9ffd-1187520239ad'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:10:35.346598 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-1bfa66f6-f226-4ef3-a2aa-c7012f387888 req-cf82d66d-1311-4665-90db-6fce39747ae2 tempest-AttachVolumeShelveTestJSON-705121323 None] attachment_update completed successfully. Oct 01 21:10:39.282169 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:10:39.375379 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Flow 'volume_create_manager' (f2ab332f-21d7-4ad4-bb84-0e685045d13e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:39.402948 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (20958e8d-12fe-4c57-9065-09ecac891d55) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:39.466842 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (20958e8d-12fe-4c57-9065-09ecac891d55) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:10:39Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-909675156',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=092a001a-07c1-4c2a-9b64-59c4a913b699,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0b5bb0c08a5145369fc64e13d190c988',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:10:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:10:39Z,use_quota=True,user_id='f74f251052ee4c9a99db0c43bc00da9f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:39.484407 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (be715002-1538-4568-ad5d-de20de107c96) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:39.506188 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (be715002-1538-4568-ad5d-de20de107c96) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:39.536693 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8ccf0756-3420-429f-80a1-a2ff9a5bec79) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:39.548748 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8ccf0756-3420-429f-80a1-a2ff9a5bec79) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '092a001a-07c1-4c2a-9b64-59c4a913b699', 'volume_name': 'volume-092a001a-07c1-4c2a-9b64-59c4a913b699', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:39.567355 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b5e56b5f-909d-4afb-b754-f441f3d8bcdb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:39.577764 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b5e56b5f-909d-4afb-b754-f441f3d8bcdb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:39.600548 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ef3c850f-c360-4288-b60c-5b6ae54e596a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:39.602034 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Volume 092a001a-07c1-4c2a-9b64-59c4a913b699: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-092a001a-07c1-4c2a-9b64-59c4a913b699', 'volume_size': 1} Oct 01 21:10:39.602717 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] creating volume 'volume-092a001a-07c1-4c2a-9b64-59c4a913b699' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:10:39.604185 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:39.749420 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:10:39.750089 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:10:39.760610 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ef3c850f-c360-4288-b60c-5b6ae54e596a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-092a001a-07c1-4c2a-9b64-59c4a913b699', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:39.773346 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c0883fe1-c7c4-40a9-8f60-c64e080942c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:39.801836 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Volume volume-092a001a-07c1-4c2a-9b64-59c4a913b699 (092a001a-07c1-4c2a-9b64-59c4a913b699): created successfully Oct 01 21:10:39.806537 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c0883fe1-c7c4-40a9-8f60-c64e080942c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:39.819564 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Flow 'volume_create_manager' (f2ab332f-21d7-4ad4-bb84-0e685045d13e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:39.842071 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-bd4cefe2-199b-4d64-b522-2b388a9574ce tempest-AttachVolumeTestJSON-1725633688 None] Created volume successfully. Oct 01 21:10:41.577336 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] Acquiring lock "cinder-5f3eb6ba-d61b-4c3c-8018-5709f993e46d-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:10:41.600336 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] Lock "cinder-5f3eb6ba-d61b-4c3c-8018-5709f993e46d-delete_volume" acquired by "delete_volume" :: waited 0.023s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:10:41.789757 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-af262b70-bcde-40d2-bbd7-858330afcfb7 req-660fa521-a5ae-44c5-bbca-6cc7070df932 tempest-AttachVolumeTestJSON-1725633688 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:10:41.911819 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:42.028921 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:10:42.029365 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] Volume volume-5f3eb6ba-d61b-4c3c-8018-5709f993e46d is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:10:42.045413 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] deleting rbd volume volume-5f3eb6ba-d61b-4c3c-8018-5709f993e46d {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:10:42.078774 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:10:42.177962 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Flow 'volume_create_manager' (c1650802-2af3-4ecc-934a-0717d085c870) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:42.194368 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7fea5145-8ae9-4179-ac9f-03425e3361a9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:42.290187 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7fea5145-8ae9-4179-ac9f-03425e3361a9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:10:41Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneNegativeTest-Volume-3228651',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=66f810f6-408a-43de-87f2-c79ad3883bdd,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='65b53a9e343e4ee08e0f9905cc895daa',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:10:42Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:10:42Z,use_quota=True,user_id='73235ec3cce94f00b9c60cd7a3be5e85',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:42.311443 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (258e44ee-0b95-4276-9e0d-2146d46381d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:42.324237 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (258e44ee-0b95-4276-9e0d-2146d46381d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:42.339278 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (375b54da-651b-46a4-98c4-42b8823dd1e7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:42.371444 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (375b54da-651b-46a4-98c4-42b8823dd1e7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '66f810f6-408a-43de-87f2-c79ad3883bdd', 'volume_name': 'volume-66f810f6-408a-43de-87f2-c79ad3883bdd', 'volume_size': 2}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:42.380049 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c9012373-ccc5-4f7f-95b7-84199363764c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:42.399239 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c9012373-ccc5-4f7f-95b7-84199363764c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:42.422270 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0af134c7-787c-433e-946d-1120f25466b6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:42.430376 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Volume 66f810f6-408a-43de-87f2-c79ad3883bdd: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-66f810f6-408a-43de-87f2-c79ad3883bdd', 'volume_size': 2} Oct 01 21:10:42.431135 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] creating volume 'volume-66f810f6-408a-43de-87f2-c79ad3883bdd' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:10:42.431977 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:42.600772 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:10:42.601917 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:10:42.605858 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0af134c7-787c-433e-946d-1120f25466b6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-66f810f6-408a-43de-87f2-c79ad3883bdd', 'volume_size': 2}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:42.624878 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a5b60891-df3a-46d6-bd5a-058171e284b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:42.637379 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=72,cinder:UPDATE=23,cinder:DELETE=2,cinder:INSERT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:42.675463 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Volume volume-66f810f6-408a-43de-87f2-c79ad3883bdd (66f810f6-408a-43de-87f2-c79ad3883bdd): created successfully Oct 01 21:10:42.690677 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a5b60891-df3a-46d6-bd5a-058171e284b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:42.696694 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Flow 'volume_create_manager' (c1650802-2af3-4ecc-934a-0717d085c870) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:42.746132 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9ec6f741-a345-43e3-a410-416ed319e3a5 tempest-VolumesCloneNegativeTest-717931020 None] Created volume successfully. Oct 01 21:10:42.787026 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] Created reservations ['d8b79da8-4e08-4831-9259-89e9d1e1b203', '265ef798-ac0d-4fa0-86fd-a0ed3fe717de', 'd7b6dfdd-3cd7-479c-b009-877b93f6ee27', 'd7f0dd8d-9a2e-4c6a-941d-5b41ebd18604'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:10:42.890170 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=57,cinder:INSERT=8,cinder:UPDATE=31,cinder:DELETE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:42.925828 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:43.017291 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:43.094839 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:10:43.104714 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] Deleted volume successfully. Oct 01 21:10:43.134918 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-85c7d8f5-7493-4888-b7b0-05e773ddf4b2 tempest-TaggedBootDevicesTest-874492856 None] Lock "cinder-5f3eb6ba-d61b-4c3c-8018-5709f993e46d-delete_volume" released by "delete_volume" :: held 1.535s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:10:43.852432 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-af262b70-bcde-40d2-bbd7-858330afcfb7 req-660fa521-a5ae-44c5-bbca-6cc7070df932 tempest-AttachVolumeTestJSON-1725633688 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 2.062s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:10:43.854614 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-af262b70-bcde-40d2-bbd7-858330afcfb7 req-660fa521-a5ae-44c5-bbca-6cc7070df932 tempest-AttachVolumeTestJSON-1725633688 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-092a001a-07c1-4c2a-9b64-59c4a913b699', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '092a001a-07c1-4c2a-9b64-59c4a913b699', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:10:43.907414 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-af262b70-bcde-40d2-bbd7-858330afcfb7 req-660fa521-a5ae-44c5-bbca-6cc7070df932 tempest-AttachVolumeTestJSON-1725633688 None] Connection info returned from driver {'name': 'volumes/volume-092a001a-07c1-4c2a-9b64-59c4a913b699', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '092a001a-07c1-4c2a-9b64-59c4a913b699', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '40c1f8b6-c2a8-4d44-8d44-f1e597a98f6f'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:10:44.085140 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-af262b70-bcde-40d2-bbd7-858330afcfb7 req-660fa521-a5ae-44c5-bbca-6cc7070df932 tempest-AttachVolumeTestJSON-1725633688 None] attachment_update completed successfully. Oct 01 21:10:44.193615 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] Acquiring lock "cinder-66f810f6-408a-43de-87f2-c79ad3883bdd-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:10:44.211036 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] Lock "cinder-66f810f6-408a-43de-87f2-c79ad3883bdd-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:10:44.287412 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] Acquiring lock "cinder-191e827c-2032-49f3-ba21-01dce9c29e11-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:10:44.377999 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] Lock "cinder-191e827c-2032-49f3-ba21-01dce9c29e11-delete_volume" acquired by "delete_volume" :: waited 0.093s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:10:44.396028 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:44.472047 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:10:44.472791 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] Volume volume-66f810f6-408a-43de-87f2-c79ad3883bdd is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:10:44.481253 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] deleting rbd volume volume-66f810f6-408a-43de-87f2-c79ad3883bdd {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:10:44.564832 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:44.713225 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:10:44.713662 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] Volume volume-191e827c-2032-49f3-ba21-01dce9c29e11 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:10:44.750230 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] deleting rbd volume volume-191e827c-2032-49f3-ba21-01dce9c29e11 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:10:44.912874 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] Created reservations ['d8aecad6-1033-4a72-8921-0750e5c84dd1', '0441977f-f953-42aa-8e85-90b8c3f827cc', 'b0e21e9e-f7ea-43bf-8332-cb83633a3e0f', '073ef263-d56d-47de-941f-7fa32849b14d'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:10:45.025481 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:45.129271 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:45.202259 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:10:45.211646 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] Deleted volume successfully. Oct 01 21:10:45.234264 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5f2f5414-b8c0-488f-8767-6086e89e41d3 tempest-VolumesCloneNegativeTest-717931020 None] Lock "cinder-66f810f6-408a-43de-87f2-c79ad3883bdd-delete_volume" released by "delete_volume" :: held 1.026s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:10:45.262694 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] Created reservations ['308bacee-cf56-4873-a7b0-332de3c8958b', '35e90061-cf0c-4d39-bf07-7a823c94b2c7', '034a823a-b550-43a3-95ff-46888cad8500', 'db409499-6912-430f-a085-e1c5ea55c250'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:10:45.396740 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:45.490614 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:45.572783 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:10:45.578791 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] Deleted volume successfully. Oct 01 21:10:45.607620 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] Acquiring lock "cinder-10008470-ec42-45af-91f0-d055c3127e46-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:10:45.643727 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] Lock "cinder-10008470-ec42-45af-91f0-d055c3127e46-delete_volume" acquired by "delete_volume" :: waited 0.036s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:10:45.677772 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-65542bd0-bd2b-40aa-b024-a46a4b21de12 tempest-TaggedBootDevicesTest-874492856 None] Lock "cinder-191e827c-2032-49f3-ba21-01dce9c29e11-delete_volume" released by "delete_volume" :: held 1.298s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:10:45.774407 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:45.917632 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:10:45.932759 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] deleting rbd volume volume-10008470-ec42-45af-91f0-d055c3127e46 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:10:46.255995 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:10:46.292763 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] deleting parent snapshot volume-10008470-ec42-45af-91f0-d055c3127e46.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:10:47.027522 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] Created reservations ['9ef9886a-eb29-4283-90bd-f77536f2e777', 'db4e557b-4ac8-4d7b-81b4-62c0d1af433c', '9e019699-62bd-419c-9c55-68cb75c917e4', 'dd39f294-9e89-4686-954c-dad4d2b3b038'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:10:47.155374 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=52,cinder:INSERT=5,cinder:UPDATE=40,cinder:DELETE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:47.164375 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:47.258276 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:47.355292 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:10:47.392750 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] Deleted volume successfully. Oct 01 21:10:47.403393 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:10:47.454565 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Flow 'volume_create_manager' (4695809f-5c62-4c5a-8b5b-113a81848b85) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:47.468842 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8dbbc47a-cc89-4746-8d15-5b44cbc86539) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:47.513532 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-672e0855-c8d1-4217-80d5-dcaed7826cf0 tempest-TaggedBootDevicesTest-874492856 None] Lock "cinder-10008470-ec42-45af-91f0-d055c3127e46-delete_volume" released by "delete_volume" :: held 1.870s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:10:47.537027 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8dbbc47a-cc89-4746-8d15-5b44cbc86539) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:10:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-826339692',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c71b7f5d-5340-4105-9d8d-c2c46de550a4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0b5bb0c08a5145369fc64e13d190c988',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:10:47Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:10:47Z,use_quota=True,user_id='f74f251052ee4c9a99db0c43bc00da9f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:47.542831 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5ffcc7df-4bf7-4388-84c2-9c5b15db6c70) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:47.546637 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5ffcc7df-4bf7-4388-84c2-9c5b15db6c70) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:47.564364 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8b2a9166-e3bd-480c-9bdd-2fb3b26a6c49) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:47.576906 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8b2a9166-e3bd-480c-9bdd-2fb3b26a6c49) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c71b7f5d-5340-4105-9d8d-c2c46de550a4', 'volume_name': 'volume-c71b7f5d-5340-4105-9d8d-c2c46de550a4', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:47.582735 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (067f80dd-66cf-41a0-b6f4-6ffa3f0dc3f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:47.597884 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (067f80dd-66cf-41a0-b6f4-6ffa3f0dc3f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:47.622804 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (269874a8-3157-4adf-a2b5-b7bfb8ee7437) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:47.625737 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Volume c71b7f5d-5340-4105-9d8d-c2c46de550a4: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c71b7f5d-5340-4105-9d8d-c2c46de550a4', 'volume_size': 1} Oct 01 21:10:47.627085 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] creating volume 'volume-c71b7f5d-5340-4105-9d8d-c2c46de550a4' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:10:47.628079 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:47.721694 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:10:47.722063 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:10:47.726342 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (269874a8-3157-4adf-a2b5-b7bfb8ee7437) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c71b7f5d-5340-4105-9d8d-c2c46de550a4', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:47.743054 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (faf305ae-4cc2-4207-9864-a55b3ab82f32) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:47.756003 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Volume volume-c71b7f5d-5340-4105-9d8d-c2c46de550a4 (c71b7f5d-5340-4105-9d8d-c2c46de550a4): created successfully Oct 01 21:10:47.759995 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (faf305ae-4cc2-4207-9864-a55b3ab82f32) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:47.771799 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Flow 'volume_create_manager' (4695809f-5c62-4c5a-8b5b-113a81848b85) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:47.803532 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6dfd969b-b93f-454b-a69c-713352bc4552 tempest-AttachVolumeTestJSON-1725633688 None] Created volume successfully. Oct 01 21:10:49.146240 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-b35ecf38-a38a-4254-bd25-64da4a28c9be req-598335ca-0d02-40d3-8cef-c00e1cfb3e21 tempest-AttachVolumeTestJSON-1725633688 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:10:50.428476 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-b35ecf38-a38a-4254-bd25-64da4a28c9be req-598335ca-0d02-40d3-8cef-c00e1cfb3e21 tempest-AttachVolumeTestJSON-1725633688 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.282s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:10:50.429374 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-b35ecf38-a38a-4254-bd25-64da4a28c9be req-598335ca-0d02-40d3-8cef-c00e1cfb3e21 tempest-AttachVolumeTestJSON-1725633688 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-c71b7f5d-5340-4105-9d8d-c2c46de550a4', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'c71b7f5d-5340-4105-9d8d-c2c46de550a4', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:10:50.471806 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-b35ecf38-a38a-4254-bd25-64da4a28c9be req-598335ca-0d02-40d3-8cef-c00e1cfb3e21 tempest-AttachVolumeTestJSON-1725633688 None] Connection info returned from driver {'name': 'volumes/volume-c71b7f5d-5340-4105-9d8d-c2c46de550a4', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'c71b7f5d-5340-4105-9d8d-c2c46de550a4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '46b68e1e-98e0-406a-b829-abed60073dfc'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:10:50.539444 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-b35ecf38-a38a-4254-bd25-64da4a28c9be req-598335ca-0d02-40d3-8cef-c00e1cfb3e21 tempest-AttachVolumeTestJSON-1725633688 None] attachment_update completed successfully. Oct 01 21:10:53.093480 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:10:53.124886 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Flow 'volume_create_manager' (67aea163-c7dd-479c-9540-3a658659337a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:53.135230 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (475f3543-1b2a-491d-a34d-30d1a17a2455) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:53.181662 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (475f3543-1b2a-491d-a34d-30d1a17a2455) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:10:53Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-2030224310',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=07294dd9-ac7c-41ee-a684-07fa45f5d09a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='24a06f5da36b4488a1b8ea01729b0e6c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:10:53Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:10:53Z,use_quota=True,user_id='902ba44d318d4a01a510f732d27467c9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:53.206412 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c9ed29ca-79c0-4cd1-a3e8-23b573e37266) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:53.225232 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c9ed29ca-79c0-4cd1-a3e8-23b573e37266) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:53.230547 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7e8a49df-ba94-4cae-9d65-76d48cc52a5e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:53.246884 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7e8a49df-ba94-4cae-9d65-76d48cc52a5e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '07294dd9-ac7c-41ee-a684-07fa45f5d09a', 'volume_name': 'volume-07294dd9-ac7c-41ee-a684-07fa45f5d09a', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:53.252150 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bae93bb6-6545-4fbe-9bec-c457bfe1cb20) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:53.256854 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bae93bb6-6545-4fbe-9bec-c457bfe1cb20) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:53.272025 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8a947e3-c6ee-45d3-9f4e-35b50ff09082) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:53.273184 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Volume 07294dd9-ac7c-41ee-a684-07fa45f5d09a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-07294dd9-ac7c-41ee-a684-07fa45f5d09a', 'volume_size': 1} Oct 01 21:10:53.273549 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] creating volume 'volume-07294dd9-ac7c-41ee-a684-07fa45f5d09a' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:10:53.274318 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:10:53.397745 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:10:53.398133 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:10:53.401744 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8a947e3-c6ee-45d3-9f4e-35b50ff09082) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-07294dd9-ac7c-41ee-a684-07fa45f5d09a', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:53.406252 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (22f07f3e-25fa-42d3-9cf7-3512a50bc24b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:10:53.421711 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Volume volume-07294dd9-ac7c-41ee-a684-07fa45f5d09a (07294dd9-ac7c-41ee-a684-07fa45f5d09a): created successfully Oct 01 21:10:53.426328 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (22f07f3e-25fa-42d3-9cf7-3512a50bc24b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:10:53.433466 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Flow 'volume_create_manager' (67aea163-c7dd-479c-9540-3a658659337a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:10:53.446228 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-72ceb515-529f-4a06-8d0a-c0d01fdce1c1 tempest-AttachVolumeNegativeTest-855754723 None] Created volume successfully. Oct 01 21:10:53.460345 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=65,cinder:UPDATE=25,cinder:INSERT=8,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:10:53.912225 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-2ea4d2af-bf7a-42cc-aa2d-052f4d53a142 req-2b0b27d0-c807-4a73-8fe1-dc7007c3af4a tempest-AttachVolumeTestJSON-1725633688 None] Terminate volume connection completed successfully. Oct 01 21:10:53.912638 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-2ea4d2af-bf7a-42cc-aa2d-052f4d53a142 req-2b0b27d0-c807-4a73-8fe1-dc7007c3af4a tempest-AttachVolumeTestJSON-1725633688 None] Deleting attachment 40c1f8b6-c2a8-4d44-8d44-f1e597a98f6f. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:10:54.839660 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-06be777c-c138-4b3d-81d4-37e2b5211061 req-c0bc5997-2a67-462b-bba5-57d76d39aa89 tempest-AttachVolumeNegativeTest-855754723 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:10:55.877772 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-ef673124-4e52-4976-961d-8d9f5764b5b0 req-5fec2595-9d68-4625-8e39-a4b704c051ae tempest-AttachVolumeTestJSON-1725633688 None] Terminate volume connection completed successfully. Oct 01 21:10:55.878550 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-ef673124-4e52-4976-961d-8d9f5764b5b0 req-5fec2595-9d68-4625-8e39-a4b704c051ae tempest-AttachVolumeTestJSON-1725633688 None] Deleting attachment 46b68e1e-98e0-406a-b829-abed60073dfc. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:10:55.947388 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-06be777c-c138-4b3d-81d4-37e2b5211061 req-c0bc5997-2a67-462b-bba5-57d76d39aa89 tempest-AttachVolumeNegativeTest-855754723 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.108s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:10:55.948732 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-06be777c-c138-4b3d-81d4-37e2b5211061 req-c0bc5997-2a67-462b-bba5-57d76d39aa89 tempest-AttachVolumeNegativeTest-855754723 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-07294dd9-ac7c-41ee-a684-07fa45f5d09a', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '07294dd9-ac7c-41ee-a684-07fa45f5d09a', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:10:56.003877 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-06be777c-c138-4b3d-81d4-37e2b5211061 req-c0bc5997-2a67-462b-bba5-57d76d39aa89 tempest-AttachVolumeNegativeTest-855754723 None] Connection info returned from driver {'name': 'volumes/volume-07294dd9-ac7c-41ee-a684-07fa45f5d09a', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '07294dd9-ac7c-41ee-a684-07fa45f5d09a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'c1890c4b-3306-4fd2-be30-bac1129624b4'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:10:56.093015 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-06be777c-c138-4b3d-81d4-37e2b5211061 req-c0bc5997-2a67-462b-bba5-57d76d39aa89 tempest-AttachVolumeNegativeTest-855754723 None] attachment_update completed successfully. Oct 01 21:11:06.082886 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=32,cinder:DELETE=1,cinder:UPDATE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:08.616247 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] Acquiring lock "cinder-c71b7f5d-5340-4105-9d8d-c2c46de550a4-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:11:08.636380 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] Lock "cinder-c71b7f5d-5340-4105-9d8d-c2c46de550a4-delete_volume" acquired by "delete_volume" :: waited 0.020s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:11:08.695587 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:08.783282 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:11:08.784405 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] Volume volume-c71b7f5d-5340-4105-9d8d-c2c46de550a4 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:11:08.801900 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] deleting rbd volume volume-c71b7f5d-5340-4105-9d8d-c2c46de550a4 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:11:09.228143 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] Created reservations ['84866af9-236b-41e2-a3d3-66c103b91667', '2665f5c3-9ce6-4bb1-93ae-16c4c4b36cd9', '640ad768-eabc-416e-9d81-629b0d9e9bbc', '360c2aac-5905-4c8e-934e-5e1be27b0597'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:09.287988 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:09.333591 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:09.399574 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:11:09.411402 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] Deleted volume successfully. Oct 01 21:11:09.435016 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-b92b6b24-d194-4293-97f5-910d951d5c96 tempest-AttachVolumeTestJSON-1725633688 None] Lock "cinder-c71b7f5d-5340-4105-9d8d-c2c46de550a4-delete_volume" released by "delete_volume" :: held 0.798s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:11:09.837160 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] Acquiring lock "cinder-092a001a-07c1-4c2a-9b64-59c4a913b699-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:11:09.858100 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] Lock "cinder-092a001a-07c1-4c2a-9b64-59c4a913b699-delete_volume" acquired by "delete_volume" :: waited 0.021s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:11:09.960353 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:10.051460 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:11:10.051933 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] Volume volume-092a001a-07c1-4c2a-9b64-59c4a913b699 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:11:10.069151 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] deleting rbd volume volume-092a001a-07c1-4c2a-9b64-59c4a913b699 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:11:10.456283 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] Created reservations ['6307899e-aa9c-4380-ab98-b690fd307564', '5a69f926-70e1-4108-b4af-4f26217886b4', 'adc7901f-0e36-428d-98b0-63d542391592', '2a6712df-5784-4c18-885f-4ba1c0e6fb93'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:10.544273 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:10.621451 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:10.725969 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:11:10.731110 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] Deleted volume successfully. Oct 01 21:11:10.753760 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5992f87f-4520-444d-afa7-d4e43f16a3a2 tempest-AttachVolumeTestJSON-1725633688 None] Lock "cinder-092a001a-07c1-4c2a-9b64-59c4a913b699-delete_volume" released by "delete_volume" :: held 0.895s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:11:11.432952 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] Acquiring lock "cinder-f05a8921-7393-4c72-bce8-e68fb45486ed-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:11:11.450449 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] Lock "cinder-f05a8921-7393-4c72-bce8-e68fb45486ed-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:11:11.527989 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:11.616677 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:11:11.617100 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] Volume volume-f05a8921-7393-4c72-bce8-e68fb45486ed is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:11:11.625934 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] deleting rbd volume volume-f05a8921-7393-4c72-bce8-e68fb45486ed {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:11:11.957256 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] Created reservations ['1d321093-635b-46bd-b04b-5240a3bc206d', '87cbed1b-65ba-478e-8caa-77f4d88dfbd8', 'cd3092f2-ed3d-40c1-9111-8314dd473764', '32c99e21-cd68-4f91-a8b4-b4139cf771f4'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:12.038352 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:12.106704 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:12.161284 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:11:12.165813 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] Deleted volume successfully. Oct 01 21:11:12.185189 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ce793e25-371b-47d9-9cfb-e9813f7105b5 tempest-AttachVolumeTestJSON-1725633688 None] Lock "cinder-f05a8921-7393-4c72-bce8-e68fb45486ed-delete_volume" released by "delete_volume" :: held 0.735s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:11:13.057985 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5fcef625-126f-4d49-aa1f-c6ef1e9bdf9d tempest-GroupsTest-13496793 None] Group tempest-Group1-233747525: creating Oct 01 21:11:13.113966 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5fcef625-126f-4d49-aa1f-c6ef1e9bdf9d tempest-GroupsTest-13496793 None] Group tempest-Group1-233747525: created successfully Oct 01 21:11:13.136426 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5fcef625-126f-4d49-aa1f-c6ef1e9bdf9d tempest-GroupsTest-13496793 None] Create group completed successfully. Oct 01 21:11:14.358372 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b95f4299-4f98-411e-a0f6-e9c2b6b96603 tempest-GroupsTest-13496793 None] Group tempest-Group2-1604826002: creating Oct 01 21:11:14.388215 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b95f4299-4f98-411e-a0f6-e9c2b6b96603 tempest-GroupsTest-13496793 None] Group tempest-Group2-1604826002: created successfully Oct 01 21:11:14.408058 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b95f4299-4f98-411e-a0f6-e9c2b6b96603 tempest-GroupsTest-13496793 None] Create group completed successfully. Oct 01 21:11:16.261262 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=67,cinder:UPDATE=26,cinder:INSERT=6,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:16.298448 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:11:16.345748 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Flow 'volume_create_manager' (a54a88bb-16dc-4ac7-b93e-30a5b1fccf52) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:11:16.354899 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9056b69c-6150-4841-8b02-404114ec04e6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:16.407528 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9056b69c-6150-4841-8b02-404114ec04e6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:11:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-volume-38345347',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=ced53037-6d3f-4e0b-8cc0-6aa3ff66b89d,host='ncdv-master-1@ceph#ceph',id=2f8595f9-e509-43b8-ab99-3cd1f5a12e0b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='06940ad486584ae391c5891858e8af7a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:11:16Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:11:16Z,use_quota=True,user_id='bf57979b45f14ca0a58a89961875f6cd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(a2819ca5-5221-46de-82ac-dcbea77b82de),volume_type_id=a2819ca5-5221-46de-82ac-dcbea77b82de)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:16.422874 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (04eb3788-7c6e-4696-9801-b50fe1859b1f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:16.429223 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (04eb3788-7c6e-4696-9801-b50fe1859b1f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:16.437372 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b31b6f67-577e-47a6-9fae-afafc1a89aec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:16.461938 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b31b6f67-577e-47a6-9fae-afafc1a89aec) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2f8595f9-e509-43b8-ab99-3cd1f5a12e0b', 'volume_name': 'volume-2f8595f9-e509-43b8-ab99-3cd1f5a12e0b', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:16.472307 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (13cc7b64-57bd-4432-9208-42d8fcfa7dbe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:16.488552 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (13cc7b64-57bd-4432-9208-42d8fcfa7dbe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:16.509843 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0bb1427b-4c6c-45aa-9327-ce9f0b20f7e9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:16.512415 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Volume 2f8595f9-e509-43b8-ab99-3cd1f5a12e0b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2f8595f9-e509-43b8-ab99-3cd1f5a12e0b', 'volume_size': 1} Oct 01 21:11:16.512904 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] creating volume 'volume-2f8595f9-e509-43b8-ab99-3cd1f5a12e0b' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:11:16.513701 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:16.679500 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:11:16.680325 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:11:16.684445 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0bb1427b-4c6c-45aa-9327-ce9f0b20f7e9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2f8595f9-e509-43b8-ab99-3cd1f5a12e0b', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:16.696015 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5c0d9d01-b19e-4b31-a997-ed66d96c46c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:16.725413 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Volume volume-2f8595f9-e509-43b8-ab99-3cd1f5a12e0b (2f8595f9-e509-43b8-ab99-3cd1f5a12e0b): created successfully Oct 01 21:11:16.735875 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5c0d9d01-b19e-4b31-a997-ed66d96c46c3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:16.766352 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Flow 'volume_create_manager' (a54a88bb-16dc-4ac7-b93e-30a5b1fccf52) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:11:16.794122 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-bcc8793e-44e7-46b9-a67d-04ef317cceeb tempest-GroupsTest-13496793 None] Created volume successfully. Oct 01 21:11:18.171166 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:18.293218 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:11:18.293751 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] Volume volume-2f8595f9-e509-43b8-ab99-3cd1f5a12e0b is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:11:18.300287 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] deleting rbd volume volume-2f8595f9-e509-43b8-ab99-3cd1f5a12e0b {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:11:18.625897 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=57,cinder:INSERT=6,cinder:UPDATE=35,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:18.647596 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] Created reservations ['fbaefd5e-be70-4de5-b6b7-3c0315df3aad'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:18.678716 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-163692935 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-163692935, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:11:18.679613 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-163692935 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-163692935, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:11:18.708137 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] Created reservations ['25accd78-8bd9-4816-bc63-a5f51a6ee885', '6bdefffd-68eb-407f-b9ed-aec4094d89db', '5e3770c6-5054-4bd9-92b1-910dfb86ff55', 'd77804b5-9f65-4f3a-885b-76f686892a65'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:18.926446 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:19.010083 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:19.070125 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:11:19.075570 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-910aac89-a397-4bd1-ba2d-5ee289f85d91 tempest-GroupsTest-13496793 None] Delete group completed successfully. Oct 01 21:11:19.709082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3d890f16-63e3-4338-90e3-73d1335704c8 tempest-GroupsTest-13496793 None] Created reservations ['8211c546-4d51-47af-9f22-b36953a12501'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:19.795824 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3d890f16-63e3-4338-90e3-73d1335704c8 tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:19.910118 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3d890f16-63e3-4338-90e3-73d1335704c8 tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:19.996840 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-3d890f16-63e3-4338-90e3-73d1335704c8 tempest-GroupsTest-13496793 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:11:20.001169 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3d890f16-63e3-4338-90e3-73d1335704c8 tempest-GroupsTest-13496793 None] Delete group completed successfully. Oct 01 21:11:21.701682 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-00b73628-a2f3-4f37-bfa3-9b0168cbd35c tempest-GroupsTest-13496793 None] Group tempest-GroupsTest-Group-1352198400: creating Oct 01 21:11:21.776230 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-00b73628-a2f3-4f37-bfa3-9b0168cbd35c tempest-GroupsTest-13496793 None] Group tempest-GroupsTest-Group-1352198400: created successfully Oct 01 21:11:21.818724 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-00b73628-a2f3-4f37-bfa3-9b0168cbd35c tempest-GroupsTest-13496793 None] Create group completed successfully. Oct 01 21:11:23.683200 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:11:23.750753 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Flow 'volume_create_manager' (855211c5-6bff-4808-9fe2-af274494aa1f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:11:23.771402 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e7e9515e-540b-44b4-965c-a3fb5902a6b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:23.801744 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:11:23.804957 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:23.947188 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e7e9515e-540b-44b4-965c-a3fb5902a6b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:11:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsTest-Volume-1716678868',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=3f8d7d04-5b78-4eab-a9c8-820e7721e528,host='ncdv-master-1@ceph#ceph',id=d4d1a418-f250-4dbb-accd-6e88b0354823,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='06940ad486584ae391c5891858e8af7a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:11:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:11:24Z,use_quota=True,user_id='bf57979b45f14ca0a58a89961875f6cd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(bf358675-3fab-468e-8d3d-58868b724af5),volume_type_id=bf358675-3fab-468e-8d3d-58868b724af5)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:23.958609 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (75b8f761-e6d5-49e4-be8d-17aae9b758cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:23.963982 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (75b8f761-e6d5-49e4-be8d-17aae9b758cc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:23.973328 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dda292f1-ce6f-474e-87c0-304d8e3d6e17) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:23.982378 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dda292f1-ce6f-474e-87c0-304d8e3d6e17) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd4d1a418-f250-4dbb-accd-6e88b0354823', 'volume_name': 'volume-d4d1a418-f250-4dbb-accd-6e88b0354823', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:24.003457 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c6a4ef67-ae47-4e91-b7d3-4a806f38d29f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:24.010530 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c6a4ef67-ae47-4e91-b7d3-4a806f38d29f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:24.021133 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4651b7e4-6bb3-4859-bf2a-79172d80f545) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:24.023594 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Volume d4d1a418-f250-4dbb-accd-6e88b0354823: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d4d1a418-f250-4dbb-accd-6e88b0354823', 'volume_size': 1} Oct 01 21:11:24.035788 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] creating volume 'volume-d4d1a418-f250-4dbb-accd-6e88b0354823' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:11:24.037370 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:24.116881 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:24.218196 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:11:24.225475 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:11:24.225852 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:11:24.229482 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4651b7e4-6bb3-4859-bf2a-79172d80f545) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d4d1a418-f250-4dbb-accd-6e88b0354823', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:24.246296 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0e101352-4bad-4d70-a639-8fcd50e85d3f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:24.286215 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Volume volume-d4d1a418-f250-4dbb-accd-6e88b0354823 (d4d1a418-f250-4dbb-accd-6e88b0354823): created successfully Oct 01 21:11:24.307242 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0e101352-4bad-4d70-a639-8fcd50e85d3f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:24.325191 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Flow 'volume_create_manager' (855211c5-6bff-4808-9fe2-af274494aa1f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:11:24.352641 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1abf48fa-8c92-4030-92bd-f73477b75177 tempest-GroupsTest-13496793 None] Created volume successfully. Oct 01 21:11:25.676598 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:11:25.701340 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Flow 'volume_create_manager' (129de971-c3e4-4834-ada4-64cdee10f791) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:11:25.711060 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (328400e2-a39e-4b87-a21b-b4d9a9254e77) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:25.730874 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (328400e2-a39e-4b87-a21b-b4d9a9254e77) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:11:25Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsTest-Volume-1856869335',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=3f8d7d04-5b78-4eab-a9c8-820e7721e528,host='ncdv-master-1@ceph#ceph',id=5e7b31ad-3551-4702-9510-039fa82aaa23,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='06940ad486584ae391c5891858e8af7a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:11:26Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:11:26Z,use_quota=True,user_id='bf57979b45f14ca0a58a89961875f6cd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(bf358675-3fab-468e-8d3d-58868b724af5),volume_type_id=bf358675-3fab-468e-8d3d-58868b724af5)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:25.747462 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9e1839a0-578e-49c2-aa5d-03eae3c07c7e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:25.753446 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9e1839a0-578e-49c2-aa5d-03eae3c07c7e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:25.767629 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (25ca77e7-6610-4193-a900-5f0983a2f1aa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:25.773546 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (25ca77e7-6610-4193-a900-5f0983a2f1aa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5e7b31ad-3551-4702-9510-039fa82aaa23', 'volume_name': 'volume-5e7b31ad-3551-4702-9510-039fa82aaa23', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:25.788328 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5a194694-b89d-4446-ac85-c61a0eee9b1f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:25.796609 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5a194694-b89d-4446-ac85-c61a0eee9b1f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:25.803204 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f9b7721e-e718-49ca-926c-c96c5ea76435) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:25.804833 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Volume 5e7b31ad-3551-4702-9510-039fa82aaa23: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5e7b31ad-3551-4702-9510-039fa82aaa23', 'volume_size': 1} Oct 01 21:11:25.805443 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] creating volume 'volume-5e7b31ad-3551-4702-9510-039fa82aaa23' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:11:25.806378 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:25.960181 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:11:25.960767 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:11:25.966483 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f9b7721e-e718-49ca-926c-c96c5ea76435) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5e7b31ad-3551-4702-9510-039fa82aaa23', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:25.976703 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ce3b6abd-509c-454f-9733-32d75db16644) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:26.003484 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Volume volume-5e7b31ad-3551-4702-9510-039fa82aaa23 (5e7b31ad-3551-4702-9510-039fa82aaa23): created successfully Oct 01 21:11:26.009068 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ce3b6abd-509c-454f-9733-32d75db16644) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:26.020733 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Flow 'volume_create_manager' (129de971-c3e4-4834-ada4-64cdee10f791) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:11:26.045218 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-15841372-d645-4729-a1ba-04a9c39db8bb tempest-GroupsTest-13496793 None] Created volume successfully. Oct 01 21:11:26.953930 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b6814e34-4dce-4e24-b198-241874f027fa tempest-GroupsTest-13496793 None] Update group completed successfully. Oct 01 21:11:28.490233 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-bfe51d41-46e8-4eaa-8fe8-d97dbd48e543 tempest-GroupsTest-13496793 None] Update group completed successfully. Oct 01 21:11:30.034928 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:30.115734 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:11:30.116375 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] Volume volume-5e7b31ad-3551-4702-9510-039fa82aaa23 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:11:30.123146 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] deleting rbd volume volume-5e7b31ad-3551-4702-9510-039fa82aaa23 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:11:30.399801 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:30.475299 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:11:30.475747 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] Volume volume-d4d1a418-f250-4dbb-accd-6e88b0354823 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:11:30.481853 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] deleting rbd volume volume-d4d1a418-f250-4dbb-accd-6e88b0354823 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:11:30.903052 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] Created reservations ['96108f1b-e892-4393-87fa-a969af4495cf'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:30.948661 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=66,cinder:UPDATE=28,cinder:DELETE=2,cinder:INSERT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:30.957666 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-1037586187 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-1037586187, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:11:30.962727 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-1037586187 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-1037586187, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:11:30.994307 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] Created reservations ['2472399b-95e3-4879-a08f-f3241764f77b', '20b244f7-ba64-41cb-baff-578d0fc9d315', 'd67fe3ac-9ac1-4ba1-adaf-4d5aabfb3bb3', '609d4e53-80dd-43af-8a09-f9580e1559d1'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:31.213228 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] Created reservations ['98ed8bef-af21-415a-a5b7-aa1693104ca7', '1f24df22-4bd2-4aed-8160-6ecef6d131a3', '9236d240-3456-4242-a289-7447f6e84cb6', '870a9ca4-85aa-40d8-88be-b7ac2efe8054'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:31.283154 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=67,cinder:INSERT=11,cinder:UPDATE=21,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:31.349963 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:31.416901 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:31.451614 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:11:31.456619 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-790296f0-eaef-4444-a6d9-fc31192a4a7a tempest-GroupsTest-13496793 None] Delete group completed successfully. Oct 01 21:11:34.121626 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-d35cfde7-20af-4434-9ca1-020dfdbefef7 req-8071e0d3-015c-47b0-99e2-f4c06587e4fb tempest-AttachVolumeShelveTestJSON-705121323 None] Terminate volume connection completed successfully. Oct 01 21:11:34.122953 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-d35cfde7-20af-4434-9ca1-020dfdbefef7 req-8071e0d3-015c-47b0-99e2-f4c06587e4fb tempest-AttachVolumeShelveTestJSON-705121323 None] Deleting attachment 05767b8f-b005-4c0c-9ffd-1187520239ad. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:11:36.611870 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a9c6f70f-a61b-4a68-9e82-d371d6522da7 tempest-GroupsV314Test-1336436521 None] Group tempest-GroupsV314Test-Group-608013349: creating Oct 01 21:11:36.640845 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a9c6f70f-a61b-4a68-9e82-d371d6522da7 tempest-GroupsV314Test-1336436521 None] Group tempest-GroupsV314Test-Group-608013349: created successfully Oct 01 21:11:36.651786 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a9c6f70f-a61b-4a68-9e82-d371d6522da7 tempest-GroupsV314Test-1336436521 None] Create group completed successfully. Oct 01 21:11:38.490395 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:11:38.514877 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Flow 'volume_create_manager' (344b1476-75b7-43db-88e2-5361e338d06b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:11:38.522554 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (eaa117ca-dd98-42fe-8ecf-11a84b2b7031) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:38.541850 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (eaa117ca-dd98-42fe-8ecf-11a84b2b7031) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:11:38Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsV314Test-Volume-290856062',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=752b34f5-b63d-4c9b-ba44-47a6ab742e2c,host='ncdv-master-1@ceph#ceph',id=809d2867-e9dc-4a18-aeaf-bddc89405072,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2e8942500e3c4be7ae908a64552336f9',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:11:38Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:11:38Z,use_quota=True,user_id='280a7a3c459a4aefa587c577ae177c61',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(cb874c1a-b607-4b25-a250-2fb57a9f47df),volume_type_id=cb874c1a-b607-4b25-a250-2fb57a9f47df)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:38.548686 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b9b1f1da-95bf-4bf5-aedf-4793dfae6061) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:38.553360 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b9b1f1da-95bf-4bf5-aedf-4793dfae6061) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:38.559055 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1b8c6bed-a5ad-4462-9831-49c3498ffc0e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:38.564099 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1b8c6bed-a5ad-4462-9831-49c3498ffc0e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '809d2867-e9dc-4a18-aeaf-bddc89405072', 'volume_name': 'volume-809d2867-e9dc-4a18-aeaf-bddc89405072', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:38.569093 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (01e57a80-1ca7-4a6c-b0c6-5d706883bbf9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:38.573817 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (01e57a80-1ca7-4a6c-b0c6-5d706883bbf9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:38.578795 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e38027c1-68e4-433d-801a-18178aac033a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:38.580498 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Volume 809d2867-e9dc-4a18-aeaf-bddc89405072: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-809d2867-e9dc-4a18-aeaf-bddc89405072', 'volume_size': 1} Oct 01 21:11:38.581155 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] creating volume 'volume-809d2867-e9dc-4a18-aeaf-bddc89405072' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:11:38.582185 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:38.735386 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:11:38.735772 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:11:38.741941 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e38027c1-68e4-433d-801a-18178aac033a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-809d2867-e9dc-4a18-aeaf-bddc89405072', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:38.756452 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5871c43e-fab8-48b4-a910-a9df53e7481c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:11:38.773931 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Volume volume-809d2867-e9dc-4a18-aeaf-bddc89405072 (809d2867-e9dc-4a18-aeaf-bddc89405072): created successfully Oct 01 21:11:38.780509 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5871c43e-fab8-48b4-a910-a9df53e7481c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:11:38.788569 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Flow 'volume_create_manager' (344b1476-75b7-43db-88e2-5361e338d06b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:11:38.802274 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-dbc4e849-fab4-47a9-9932-3cca38946f2e tempest-GroupsV314Test-1336436521 None] Created volume successfully. Oct 01 21:11:40.979592 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9e0e84fc-6224-4ad9-b2ab-cdbdba810fd7 tempest-GroupsV314Test-1336436521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:41.063036 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9e0e84fc-6224-4ad9-b2ab-cdbdba810fd7 tempest-GroupsV314Test-1336436521 None] creating snapshot='volume-f7faf0f1-4ea5-43d0-b4be-19ac72fb7aba.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:11:41.265917 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9e0e84fc-6224-4ad9-b2ab-cdbdba810fd7 tempest-GroupsV314Test-1336436521 None] cloning 'volume-809d2867-e9dc-4a18-aeaf-bddc89405072@volume-f7faf0f1-4ea5-43d0-b4be-19ac72fb7aba.clone_snap' to 'volume-f7faf0f1-4ea5-43d0-b4be-19ac72fb7aba' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:11:41.459759 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9e0e84fc-6224-4ad9-b2ab-cdbdba810fd7 tempest-GroupsV314Test-1336436521 None] Volume volume-809d2867-e9dc-4a18-aeaf-bddc89405072 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:11:41.465725 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9e0e84fc-6224-4ad9-b2ab-cdbdba810fd7 tempest-GroupsV314Test-1336436521 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:11:41.466414 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9e0e84fc-6224-4ad9-b2ab-cdbdba810fd7 tempest-GroupsV314Test-1336436521 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:11:41.506146 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9e0e84fc-6224-4ad9-b2ab-cdbdba810fd7 tempest-GroupsV314Test-1336436521 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:11:41.596806 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9e0e84fc-6224-4ad9-b2ab-cdbdba810fd7 tempest-GroupsV314Test-1336436521 None] Create group from source-group-752b34f5-b63d-4c9b-ba44-47a6ab742e2c completed successfully. Oct 01 21:11:43.005490 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:43.097943 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:11:43.105451 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] deleting rbd volume volume-f7faf0f1-4ea5-43d0-b4be-19ac72fb7aba {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:11:43.349489 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:11:43.395463 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] Volume volume-809d2867-e9dc-4a18-aeaf-bddc89405072 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:11:43.396483 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] deleting parent snapshot volume-f7faf0f1-4ea5-43d0-b4be-19ac72fb7aba.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:11:44.319058 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] Created reservations ['5ba17081-186c-419f-8aed-d9e3bd28364e'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:44.336735 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] Deprecated: Default quota for resource: volumes_tempest-GroupsV314Test-volume-type-389404 is set by the default quota flag: quota_volumes_tempest-GroupsV314Test-volume-type-389404, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:11:44.337625 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsV314Test-volume-type-389404 is set by the default quota flag: quota_gigabytes_tempest-GroupsV314Test-volume-type-389404, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:11:44.352084 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] Created reservations ['9eedfa9a-50f5-4e56-bb57-ce616829dd15', '847b4a40-214c-412b-8b9b-c38ec1a12cc7', '24104562-31ca-4598-9450-076786bb5dcc', '9d574143-3047-4b16-b151-c58e26045fc8'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:44.391503 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=59,cinder:UPDATE=38,cinder:INSERT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:44.432901 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:44.472825 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:44.534588 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:11:44.539566 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-dc233368-4b31-4d80-9dce-fa38566e36e1 tempest-GroupsV314Test-1336436521 None] Delete group completed successfully. Oct 01 21:11:45.897671 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15e667fd-c46f-4914-870b-6df642654ccf tempest-GroupsV314Test-1336436521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:46.016096 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15e667fd-c46f-4914-870b-6df642654ccf tempest-GroupsV314Test-1336436521 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:11:46.017892 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15e667fd-c46f-4914-870b-6df642654ccf tempest-GroupsV314Test-1336436521 None] Volume volume-809d2867-e9dc-4a18-aeaf-bddc89405072 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:11:46.036379 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15e667fd-c46f-4914-870b-6df642654ccf tempest-GroupsV314Test-1336436521 None] deleting rbd volume volume-809d2867-e9dc-4a18-aeaf-bddc89405072 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:11:46.081752 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-2f3b89c6-7d0f-4ecc-9c42-4e503136e113 tempest-ServersTestBootFromVolume-1669384068 None] Terminate volume connection completed successfully. Oct 01 21:11:46.086757 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-2f3b89c6-7d0f-4ecc-9c42-4e503136e113 tempest-ServersTestBootFromVolume-1669384068 None] Deleting attachment 4846d155-cf33-4dd7-9129-cc3721415614. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:11:46.433017 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] Acquiring lock "cinder-057ca534-4250-412a-aa43-66638a05be1b-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:11:46.492236 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] Lock "cinder-057ca534-4250-412a-aa43-66638a05be1b-delete_volume" acquired by "delete_volume" :: waited 0.059s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:11:46.626197 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-15e667fd-c46f-4914-870b-6df642654ccf tempest-GroupsV314Test-1336436521 None] Created reservations ['6a20293f-9cde-42c0-9780-59924a5db207'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:46.707499 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:46.844573 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:11:46.864744 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] deleting rbd volume volume-057ca534-4250-412a-aa43-66638a05be1b {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:11:46.931926 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-15e667fd-c46f-4914-870b-6df642654ccf tempest-GroupsV314Test-1336436521 None] Created reservations ['d81d2041-5a1e-40f6-b08f-c0cc0e33baa6', '3df28c7e-6193-436a-b9ae-0eedbd6025b0', '8222189b-50ea-4a5c-a271-b480caa14e93', '31b3ee39-c13e-4aa7-824e-eb7fdcca3723'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:47.132936 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15e667fd-c46f-4914-870b-6df642654ccf tempest-GroupsV314Test-1336436521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:47.211551 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15e667fd-c46f-4914-870b-6df642654ccf tempest-GroupsV314Test-1336436521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:47.299517 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-15e667fd-c46f-4914-870b-6df642654ccf tempest-GroupsV314Test-1336436521 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:11:47.302254 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:11:47.344828 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] deleting parent snapshot volume-057ca534-4250-412a-aa43-66638a05be1b.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:11:48.383834 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-15e667fd-c46f-4914-870b-6df642654ccf tempest-GroupsV314Test-1336436521 None] Delete group completed successfully. Oct 01 21:11:48.455461 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] Created reservations ['2e06e10d-c79d-4234-88e3-a2f2da789023', '19b8d29c-aabd-420f-8b7c-a7df0876afea', 'dc7e4923-ea9f-4174-af7b-cb29bdf666a8', 'a6f3ae74-38f2-4325-b495-4e7338b697d6'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:11:48.478963 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=31,cinder:SELECT=63,cinder:DELETE=1,cinder:INSERT=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:11:48.565815 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:48.649040 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:11:48.718969 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:11:48.731990 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] Deleted volume successfully. Oct 01 21:11:48.760585 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-ac7ae38f-d33e-41cf-97e6-67af8efd0e43 req-ea41654c-269b-4002-81fc-943bb27a7c55 tempest-ServersTestBootFromVolume-1669384068 None] Lock "cinder-057ca534-4250-412a-aa43-66638a05be1b-delete_volume" released by "delete_volume" :: held 2.267s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:11:58.742247 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=20,cinder:SELECT=45,cinder:INSERT=6,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:01.757047 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:12:01.821186 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Flow 'volume_create_manager' (8ec8e3a4-122e-4aa3-981a-9485041013b4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:12:01.842737 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (204f52e5-d49a-47d3-9a53-2ce4545809a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:01.902260 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (204f52e5-d49a-47d3-9a53-2ce4545809a8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:12:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeTypesTest-volume-1489493816',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d5393b5b-0c97-428b-b330-67a1cd5c5112,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6034100b31904b83870cd7da3e8dfc0c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:12:02Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:12:02Z,use_quota=True,user_id='40f73d17e08d4ca58a70fa38024fc78d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(081c60e4-ff23-4ac6-9b3a-fa93954cfcc1),volume_type_id=081c60e4-ff23-4ac6-9b3a-fa93954cfcc1)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:01.917556 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8f976ff3-3195-474b-a42c-fdc25f4b6417) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:01.927687 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8f976ff3-3195-474b-a42c-fdc25f4b6417) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:01.941181 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2724aa70-90dd-4919-a3b1-5a0249be8f75) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:01.949774 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2724aa70-90dd-4919-a3b1-5a0249be8f75) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd5393b5b-0c97-428b-b330-67a1cd5c5112', 'volume_name': 'volume-d5393b5b-0c97-428b-b330-67a1cd5c5112', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:01.959431 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d8f40cff-8bf1-4437-a91a-51846ed4ce39) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:01.965426 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d8f40cff-8bf1-4437-a91a-51846ed4ce39) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:01.972272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8726923a-ad10-4611-a8a3-350294c34154) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:01.973867 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Volume d5393b5b-0c97-428b-b330-67a1cd5c5112: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d5393b5b-0c97-428b-b330-67a1cd5c5112', 'volume_size': 1} Oct 01 21:12:01.974526 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] creating volume 'volume-d5393b5b-0c97-428b-b330-67a1cd5c5112' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:12:01.976232 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:02.148916 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] extra_specs: {'storage_protocol': 'ceph', 'vendor_name': 'Open Source'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:12:02.149786 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] extra_specs: {'storage_protocol': 'ceph', 'vendor_name': 'Open Source'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:12:02.158905 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8726923a-ad10-4611-a8a3-350294c34154) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d5393b5b-0c97-428b-b330-67a1cd5c5112', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:02.167338 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (620ea986-2aa6-4157-b589-a710dfb04653) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:02.188808 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Volume volume-d5393b5b-0c97-428b-b330-67a1cd5c5112 (d5393b5b-0c97-428b-b330-67a1cd5c5112): created successfully Oct 01 21:12:02.206833 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (620ea986-2aa6-4157-b589-a710dfb04653) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:02.215639 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Flow 'volume_create_manager' (8ec8e3a4-122e-4aa3-981a-9485041013b4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:12:02.235134 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ffc4b6f3-cdc0-410a-9bda-310a0d527779 tempest-VolumeTypesTest-794767651 None] Created volume successfully. Oct 01 21:12:03.227002 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7eaa6d2d-d59d-43b3-8a81-b9db4f99f099 tempest-VolumeTypesTest-794767651 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:03.306220 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7eaa6d2d-d59d-43b3-8a81-b9db4f99f099 tempest-VolumeTypesTest-794767651 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:03.402184 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-7eaa6d2d-d59d-43b3-8a81-b9db4f99f099 tempest-VolumeTypesTest-794767651 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:12:03.413533 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7eaa6d2d-d59d-43b3-8a81-b9db4f99f099 tempest-VolumeTypesTest-794767651 None] Retype volume completed successfully. Oct 01 21:12:05.804455 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] Acquiring lock "cinder-d5393b5b-0c97-428b-b330-67a1cd5c5112-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:12:05.824067 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] Lock "cinder-d5393b5b-0c97-428b-b330-67a1cd5c5112-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:12:05.902894 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:05.992470 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:12:05.993301 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] Volume volume-d5393b5b-0c97-428b-b330-67a1cd5c5112 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:12:06.015921 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] deleting rbd volume volume-d5393b5b-0c97-428b-b330-67a1cd5c5112 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:12:06.403688 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-264020316 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-264020316, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:12:06.405419 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-264020316 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-264020316, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:12:06.427049 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] Created reservations ['46c01fa5-2f8f-46cb-9198-264df03d530a', '52dcf405-66d5-41a8-a61c-ccd8e4329116', '7a48774f-82fb-4091-86a4-6fc4f6a14104', '6e21632f-caac-45ed-95fc-11edee6129a9'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:12:06.529464 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:06.600808 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:06.671444 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:12:06.681612 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] Deleted volume successfully. Oct 01 21:12:06.708239 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c82be803-c99e-40a1-8d07-5d4d183e6dc5 tempest-VolumeTypesTest-794767651 None] Lock "cinder-d5393b5b-0c97-428b-b330-67a1cd5c5112-delete_volume" released by "delete_volume" :: held 0.885s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:12:13.108697 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-eb958ee2-35a8-4059-872c-7485c7e6bf45 req-dc4cecc8-232c-405e-92cd-013dbedfff97 tempest-AttachVolumeNegativeTest-855754723 None] Terminate volume connection completed successfully. Oct 01 21:12:13.109310 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-eb958ee2-35a8-4059-872c-7485c7e6bf45 req-dc4cecc8-232c-405e-92cd-013dbedfff97 tempest-AttachVolumeNegativeTest-855754723 None] Deleting attachment c1890c4b-3306-4fd2-be30-bac1129624b4. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:12:23.805712 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:12:23.807102 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:23.881579 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:23.961986 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:12:24.290164 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=24,cinder:SELECT=40 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:34.297393 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=32,cinder:UPDATE=15,cinder:DELETE=2,cinder:INSERT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:37.335624 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:12:37.408266 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Flow 'volume_create_manager' (4c19b8f8-3907-4211-a73d-5000328ad5bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:12:37.439842 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3b23c988-70dc-4b00-b21c-152f9c81112d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:37.468006 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3b23c988-70dc-4b00-b21c-152f9c81112d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:12:37Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-589572517',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e961484c-2fba-4c14-8029-87f4dd61959d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fa37082733634ec688c9d6272a0d9b1d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:12:37Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:12:37Z,use_quota=True,user_id='8d79dff848bf40fc89377e13736c527b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:37.478239 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ade46725-75d3-4c3a-bd04-37781dafb1ac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:37.488611 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ade46725-75d3-4c3a-bd04-37781dafb1ac) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:37.496772 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d9f8a7c8-6547-4612-8498-af3fd69aec0e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:37.500914 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d9f8a7c8-6547-4612-8498-af3fd69aec0e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e961484c-2fba-4c14-8029-87f4dd61959d', 'volume_name': 'volume-e961484c-2fba-4c14-8029-87f4dd61959d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:37.505931 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (46a426a9-5813-4906-9868-bf43b24059bd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:37.509787 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (46a426a9-5813-4906-9868-bf43b24059bd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:37.514479 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (86ba862b-ff81-4f93-a00d-e4fcb164b80d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:37.515876 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Volume e961484c-2fba-4c14-8029-87f4dd61959d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e961484c-2fba-4c14-8029-87f4dd61959d', 'volume_size': 1} Oct 01 21:12:37.516495 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] creating volume 'volume-e961484c-2fba-4c14-8029-87f4dd61959d' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:12:37.517489 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:37.611609 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:12:37.611985 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:12:37.615566 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (86ba862b-ff81-4f93-a00d-e4fcb164b80d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e961484c-2fba-4c14-8029-87f4dd61959d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:37.620738 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ceeaae47-4ed6-4a79-bdbc-575a2cb5d1c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:37.638406 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Volume volume-e961484c-2fba-4c14-8029-87f4dd61959d (e961484c-2fba-4c14-8029-87f4dd61959d): created successfully Oct 01 21:12:37.642384 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ceeaae47-4ed6-4a79-bdbc-575a2cb5d1c5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:37.649211 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Flow 'volume_create_manager' (4c19b8f8-3907-4211-a73d-5000328ad5bf) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:12:37.681859 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-43712809-a22f-491f-8f32-792a74c1dd77 tempest-AttachVolumeShelveTestJSON-705121323 None] Created volume successfully. Oct 01 21:12:44.501957 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:12:44.522141 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (62f287ec-25eb-4371-93db-ed9b72fb81b5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:12:44.528565 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2e4f1251-336d-4854-ab18-87e72ea0da5c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:44.547043 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2e4f1251-336d-4854-ab18-87e72ea0da5c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:12:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-71193679',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=642746eb-e0e8-4aed-8b8d-67e3230dbcb8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='55864f798646443bbdfe30f8b9fbefb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:12:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:12:44Z,use_quota=True,user_id='78cc9dbf1b344a35b9be09ad32549a62',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:44.552491 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e2e25a80-32e7-4b42-9dcf-aaac2ff0ee33) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:44.556484 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e2e25a80-32e7-4b42-9dcf-aaac2ff0ee33) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:44.562228 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (107d0100-fefe-47e0-b5df-659c3f065578) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:44.566258 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (107d0100-fefe-47e0-b5df-659c3f065578) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '642746eb-e0e8-4aed-8b8d-67e3230dbcb8', 'volume_name': 'volume-642746eb-e0e8-4aed-8b8d-67e3230dbcb8', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:44.571024 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (73d1a0fc-6d21-49f0-9279-01a389c0aea5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:44.574808 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (73d1a0fc-6d21-49f0-9279-01a389c0aea5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:44.579309 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (de8491f8-d451-42e1-85a8-bc9fc57929ea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:44.580351 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Volume 642746eb-e0e8-4aed-8b8d-67e3230dbcb8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-642746eb-e0e8-4aed-8b8d-67e3230dbcb8', 'volume_size': 1} Oct 01 21:12:44.580746 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] creating volume 'volume-642746eb-e0e8-4aed-8b8d-67e3230dbcb8' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:12:44.581391 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:44.634406 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:12:44.634818 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:12:44.638830 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (de8491f8-d451-42e1-85a8-bc9fc57929ea) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-642746eb-e0e8-4aed-8b8d-67e3230dbcb8', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:44.643363 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4008fb99-01ad-44a7-a57e-0e7e73e9ec62) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:44.654790 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Volume volume-642746eb-e0e8-4aed-8b8d-67e3230dbcb8 (642746eb-e0e8-4aed-8b8d-67e3230dbcb8): created successfully Oct 01 21:12:44.659920 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4008fb99-01ad-44a7-a57e-0e7e73e9ec62) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:44.665615 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (62f287ec-25eb-4371-93db-ed9b72fb81b5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:12:44.678291 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d741373c-a09e-4c4b-bafc-7189c30df4fd tempest-VolumesActionsTest-1835404292 None] Created volume successfully. Oct 01 21:12:45.681741 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7bac1190-2786-4bab-b75b-2140d89c539f tempest-VolumesActionsTest-1835404292 None] Acquiring lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8" by "attach_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:12:45.695294 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7bac1190-2786-4bab-b75b-2140d89c539f tempest-VolumesActionsTest-1835404292 None] Lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8" acquired by "attach_volume" :: waited 0.013s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:12:45.790396 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7bac1190-2786-4bab-b75b-2140d89c539f tempest-VolumesActionsTest-1835404292 None] Attaching volume 642746eb-e0e8-4aed-8b8d-67e3230dbcb8 to instance e7ddf335-7d36-4c99-959e-1d281a427fb4 at mountpoint /dev/sdb on host None. Oct 01 21:12:45.844986 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7bac1190-2786-4bab-b75b-2140d89c539f tempest-VolumesActionsTest-1835404292 None] Attach volume completed successfully. Oct 01 21:12:45.866930 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7bac1190-2786-4bab-b75b-2140d89c539f tempest-VolumesActionsTest-1835404292 None] Lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8" released by "attach_volume" :: held 0.172s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:12:46.618408 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3f4309a0-9ed8-4ac9-b06a-5343018ffebb tempest-VolumesActionsTest-679890519 None] Terminate volume connection completed successfully. Oct 01 21:12:46.655030 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3f4309a0-9ed8-4ac9-b06a-5343018ffebb tempest-VolumesActionsTest-679890519 None] Acquiring lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8-detach_volume" by "detach_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:12:46.680590 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3f4309a0-9ed8-4ac9-b06a-5343018ffebb tempest-VolumesActionsTest-679890519 None] Lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8-detach_volume" acquired by "detach_volume" :: waited 0.026s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:12:46.701641 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3f4309a0-9ed8-4ac9-b06a-5343018ffebb tempest-VolumesActionsTest-679890519 None] Detaching volume 642746eb-e0e8-4aed-8b8d-67e3230dbcb8 from instance e7ddf335-7d36-4c99-959e-1d281a427fb4. Oct 01 21:12:46.795649 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3f4309a0-9ed8-4ac9-b06a-5343018ffebb tempest-VolumesActionsTest-679890519 None] Detach volume completed successfully. Oct 01 21:12:46.804431 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3f4309a0-9ed8-4ac9-b06a-5343018ffebb tempest-VolumesActionsTest-679890519 None] Lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8-detach_volume" released by "detach_volume" :: held 0.124s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:12:47.151989 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-b3a72977-d252-405e-bdb2-b1a80aeec9fc tempest-VolumesActionsTest-1835404292 None] Acquiring lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8-detach_volume" by "detach_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:12:47.183124 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-b3a72977-d252-405e-bdb2-b1a80aeec9fc tempest-VolumesActionsTest-1835404292 None] Lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8-detach_volume" acquired by "detach_volume" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:12:47.184320 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b3a72977-d252-405e-bdb2-b1a80aeec9fc tempest-VolumesActionsTest-1835404292 None] Volume detach called, but volume not attached. Oct 01 21:12:47.249696 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-b3a72977-d252-405e-bdb2-b1a80aeec9fc tempest-VolumesActionsTest-1835404292 None] Lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8-detach_volume" released by "detach_volume" :: held 0.088s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:12:54.486845 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=28,cinder:UPDATE=4,cinder:DELETE=2,cinder:INSERT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:54.513890 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:12:54.562937 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (31dcab29-d77b-44f6-8862-f59815d4a6a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:12:54.582242 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8ea562db-54a8-429c-a573-676eefb0e33f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:54.627747 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8ea562db-54a8-429c-a573-676eefb0e33f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:12:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1955944621',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=61a5b77f-aa6f-40c6-9ed8-6eaf759e777d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='55864f798646443bbdfe30f8b9fbefb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:12:54Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:12:54Z,use_quota=True,user_id='78cc9dbf1b344a35b9be09ad32549a62',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:54.638113 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (681e14e2-c430-45ca-85ba-764090e42a2e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:54.647329 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (681e14e2-c430-45ca-85ba-764090e42a2e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:54.661412 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (790347ed-1408-42e7-b307-bcf50ec13b54) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:54.665760 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (790347ed-1408-42e7-b307-bcf50ec13b54) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '61a5b77f-aa6f-40c6-9ed8-6eaf759e777d', 'volume_name': 'volume-61a5b77f-aa6f-40c6-9ed8-6eaf759e777d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:54.683871 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3925f4fc-872e-458c-8fd8-449ad47cb8f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:54.697934 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3925f4fc-872e-458c-8fd8-449ad47cb8f9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:54.711177 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dd5a2353-89d7-4e88-b2bf-ed5b2d9b12c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:54.712595 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Volume 61a5b77f-aa6f-40c6-9ed8-6eaf759e777d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-61a5b77f-aa6f-40c6-9ed8-6eaf759e777d', 'volume_size': 1} Oct 01 21:12:54.715778 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] creating volume 'volume-61a5b77f-aa6f-40c6-9ed8-6eaf759e777d' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:12:54.716516 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:54.824379 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:12:54.825028 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:12:54.829469 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dd5a2353-89d7-4e88-b2bf-ed5b2d9b12c3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-61a5b77f-aa6f-40c6-9ed8-6eaf759e777d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:54.840683 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ecf1120c-c730-4421-909a-311d60906180) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:54.857506 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Volume volume-61a5b77f-aa6f-40c6-9ed8-6eaf759e777d (61a5b77f-aa6f-40c6-9ed8-6eaf759e777d): created successfully Oct 01 21:12:54.867036 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ecf1120c-c730-4421-909a-311d60906180) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:54.883694 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (31dcab29-d77b-44f6-8862-f59815d4a6a8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:12:54.903436 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a773097b-3952-4444-ae3f-45addacc62e7 tempest-VolumesActionsTest-1835404292 None] Created volume successfully. Oct 01 21:12:55.812164 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] Acquiring lock "cinder-61a5b77f-aa6f-40c6-9ed8-6eaf759e777d-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:12:55.837855 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] Lock "cinder-61a5b77f-aa6f-40c6-9ed8-6eaf759e777d-delete_volume" acquired by "delete_volume" :: waited 0.025s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:12:55.942199 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:56.071524 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:12:56.072289 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] Volume volume-61a5b77f-aa6f-40c6-9ed8-6eaf759e777d is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:12:56.086319 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] deleting rbd volume volume-61a5b77f-aa6f-40c6-9ed8-6eaf759e777d {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:12:56.527454 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] Created reservations ['e2e013fe-0dcb-4c11-bcca-813305664cc2', '0dc13900-2f9c-4ebb-b23c-4143a0b0f7c8', '489ade9f-f593-4079-bdcc-c1c13e5c3c93', '1ce72ac0-cf51-47a0-a035-b51c0a7d452f'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:12:56.602593 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:56.688363 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:56.757551 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:12:56.762144 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] Deleted volume successfully. Oct 01 21:12:56.791347 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-41ab89aa-5b88-40fa-8ddf-cc7f698aba25 tempest-VolumesActionsTest-679890519 None] Lock "cinder-61a5b77f-aa6f-40c6-9ed8-6eaf759e777d-delete_volume" released by "delete_volume" :: held 0.956s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:12:57.742072 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:12:57.774967 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (adc59794-a998-4252-bd1f-b191ee123b1a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:12:57.791129 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c934c74c-7171-4181-8e3d-cba90afc889d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:57.818724 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c934c74c-7171-4181-8e3d-cba90afc889d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:12:57Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1582625256',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=313e0eaf-c687-4112-a9bd-0ec5142040d0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='55864f798646443bbdfe30f8b9fbefb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:12:58Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:12:58Z,use_quota=True,user_id='78cc9dbf1b344a35b9be09ad32549a62',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:57.830332 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ff5fbd7e-b7e0-40fe-8f8a-637aa90daae7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:57.836001 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ff5fbd7e-b7e0-40fe-8f8a-637aa90daae7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:57.842429 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (137d303f-3c39-4a28-81eb-a1a9e3c6aab7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:57.847725 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (137d303f-3c39-4a28-81eb-a1a9e3c6aab7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '313e0eaf-c687-4112-a9bd-0ec5142040d0', 'volume_name': 'volume-313e0eaf-c687-4112-a9bd-0ec5142040d0', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:57.858082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c462b430-c8f6-4e26-815b-dc3464520ff5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:57.862742 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c462b430-c8f6-4e26-815b-dc3464520ff5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:57.868134 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5295319c-8809-4021-be65-472d8e351ace) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:57.869624 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Volume 313e0eaf-c687-4112-a9bd-0ec5142040d0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-313e0eaf-c687-4112-a9bd-0ec5142040d0', 'volume_size': 1} Oct 01 21:12:57.870288 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] creating volume 'volume-313e0eaf-c687-4112-a9bd-0ec5142040d0' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:12:57.871405 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:57.996812 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:12:57.997701 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:12:58.005788 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5295319c-8809-4021-be65-472d8e351ace) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-313e0eaf-c687-4112-a9bd-0ec5142040d0', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:58.018292 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d8567a69-f98e-425b-983f-6f25020f193a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:12:58.044641 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Volume volume-313e0eaf-c687-4112-a9bd-0ec5142040d0 (313e0eaf-c687-4112-a9bd-0ec5142040d0): created successfully Oct 01 21:12:58.052096 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d8567a69-f98e-425b-983f-6f25020f193a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:12:58.072024 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (adc59794-a998-4252-bd1f-b191ee123b1a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:12:58.091995 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5c3d2ff1-be7b-4ebc-966e-1409b009e52f tempest-VolumesActionsTest-1835404292 None] Created volume successfully. Oct 01 21:12:59.178331 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] Acquiring lock "cinder-313e0eaf-c687-4112-a9bd-0ec5142040d0-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:12:59.199995 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] Lock "cinder-313e0eaf-c687-4112-a9bd-0ec5142040d0-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:12:59.265070 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:12:59.350660 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:12:59.351438 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] Volume volume-313e0eaf-c687-4112-a9bd-0ec5142040d0 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:12:59.359124 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] deleting rbd volume volume-313e0eaf-c687-4112-a9bd-0ec5142040d0 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:12:59.769397 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] Created reservations ['e866f565-6e18-4708-8a2d-0c987575da59', '1a11b718-950a-4cff-a3ab-5a69d40e8bec', 'b19a4058-ce2e-46f9-8ad2-2dad7bea9e40', '3c34df7a-879a-4076-8893-52e1768b8c02'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:12:59.906029 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=63,cinder:UPDATE=30,cinder:INSERT=6,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:12:59.915575 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:00.072121 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:00.169502 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:00.174374 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] Deleted volume successfully. Oct 01 21:13:00.192314 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-90ee6534-f202-47a1-af46-1d59f5aa92b7 tempest-VolumesActionsTest-679890519 None] Lock "cinder-313e0eaf-c687-4112-a9bd-0ec5142040d0-delete_volume" released by "delete_volume" :: held 0.992s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:00.323627 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:00.384406 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Flow 'volume_create_manager' (1ed1121a-fe61-4ba6-9cdd-66c827dbd657) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:00.403433 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fd3caa6d-fa83-4ff6-8d9e-9b0dcdec5378) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:00.443663 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fd3caa6d-fa83-4ff6-8d9e-9b0dcdec5378) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:00Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesServicesTestJSON-Volume-1372483471',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d91bc858-3e21-4ae4-9e00-a8b731207dbb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='62aa9b4263524351843f7136dd44a9d0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:00Z,use_quota=True,user_id='b38792f636ab4f36abce0bf90faaef80',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:00.457327 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7ab36887-4f98-4f4a-ae1a-8007462d7695) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:00.460688 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7ab36887-4f98-4f4a-ae1a-8007462d7695) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:00.471097 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2b3a3bea-6226-4009-9274-97020d66cb4e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:00.484667 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2b3a3bea-6226-4009-9274-97020d66cb4e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd91bc858-3e21-4ae4-9e00-a8b731207dbb', 'volume_name': 'volume-d91bc858-3e21-4ae4-9e00-a8b731207dbb', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:00.492190 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ef7c95cd-fd34-439a-9337-a56d60ba4224) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:00.508760 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ef7c95cd-fd34-439a-9337-a56d60ba4224) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:00.513562 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e634eb1b-dcf7-4b81-bd50-e45c8275f1ce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:00.519591 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Volume d91bc858-3e21-4ae4-9e00-a8b731207dbb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d91bc858-3e21-4ae4-9e00-a8b731207dbb', 'volume_size': 1} Oct 01 21:13:00.522631 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] creating volume 'volume-d91bc858-3e21-4ae4-9e00-a8b731207dbb' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:00.522631 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:00.647185 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:00.647561 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:00.651214 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e634eb1b-dcf7-4b81-bd50-e45c8275f1ce) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d91bc858-3e21-4ae4-9e00-a8b731207dbb', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:00.656235 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2b5a8e80-0616-4804-8ca4-3a3b2a7ca4d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:00.682817 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Volume volume-d91bc858-3e21-4ae4-9e00-a8b731207dbb (d91bc858-3e21-4ae4-9e00-a8b731207dbb): created successfully Oct 01 21:13:00.691691 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2b5a8e80-0616-4804-8ca4-3a3b2a7ca4d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:00.705030 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Flow 'volume_create_manager' (1ed1121a-fe61-4ba6-9cdd-66c827dbd657) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:00.726280 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8173cc3f-642f-4bec-b82c-81563c5781f5 tempest-VolumesServicesTestJSON-1055956945 None] Created volume successfully. Oct 01 21:13:01.227719 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:01.308706 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (a2750b87-8e37-4f70-a121-109c104e8dca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:01.344315 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (58eda9ce-2199-40c9-955f-1bf31a331d7a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:01.408013 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (58eda9ce-2199-40c9-955f-1bf31a331d7a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-176418864',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2c4d0b33-bffa-4227-94e9-5dcb58c177bf,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='55864f798646443bbdfe30f8b9fbefb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:01Z,use_quota=True,user_id='78cc9dbf1b344a35b9be09ad32549a62',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:01.419805 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (03d379b7-f4f0-43cd-b99f-1e4f3334d842) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:01.426034 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (03d379b7-f4f0-43cd-b99f-1e4f3334d842) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:01.435618 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dbe850fe-bfa3-4a0e-a4e8-8493152b5d77) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:01.455717 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dbe850fe-bfa3-4a0e-a4e8-8493152b5d77) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2c4d0b33-bffa-4227-94e9-5dcb58c177bf', 'volume_name': 'volume-2c4d0b33-bffa-4227-94e9-5dcb58c177bf', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:01.461817 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a154fc5f-8b70-4223-8597-30dce8c297b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:01.465864 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a154fc5f-8b70-4223-8597-30dce8c297b9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:01.473083 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0f74dbd5-fdea-431a-ab2d-539e2ad4aeb4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:01.474247 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Volume 2c4d0b33-bffa-4227-94e9-5dcb58c177bf: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2c4d0b33-bffa-4227-94e9-5dcb58c177bf', 'volume_size': 1} Oct 01 21:13:01.474732 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] creating volume 'volume-2c4d0b33-bffa-4227-94e9-5dcb58c177bf' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:01.476421 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:01.608382 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] Acquiring lock "cinder-d91bc858-3e21-4ae4-9e00-a8b731207dbb-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:01.622083 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] Lock "cinder-d91bc858-3e21-4ae4-9e00-a8b731207dbb-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:01.647755 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:01.648290 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:01.655373 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0f74dbd5-fdea-431a-ab2d-539e2ad4aeb4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2c4d0b33-bffa-4227-94e9-5dcb58c177bf', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:01.669675 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bbc43637-9040-440e-b8f4-94bfd758c9e6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:01.734873 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Volume volume-2c4d0b33-bffa-4227-94e9-5dcb58c177bf (2c4d0b33-bffa-4227-94e9-5dcb58c177bf): created successfully Oct 01 21:13:01.739842 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bbc43637-9040-440e-b8f4-94bfd758c9e6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:01.747766 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (a2750b87-8e37-4f70-a121-109c104e8dca) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:01.843364 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9c987c04-601e-40bf-9c6e-eee96411beeb tempest-VolumesActionsTest-1835404292 None] Created volume successfully. Oct 01 21:13:01.899763 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:02.054459 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:02.054952 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] Volume volume-d91bc858-3e21-4ae4-9e00-a8b731207dbb is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:02.065088 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] deleting rbd volume volume-d91bc858-3e21-4ae4-9e00-a8b731207dbb {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:02.164599 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:02.188563 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Flow 'volume_create_manager' (ef03c92b-40b1-4f77-9a6b-a2e290675941) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:02.196949 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ddc5bc36-ba1f-4d89-af12-7c0e04fe59a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:02.216812 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ddc5bc36-ba1f-4d89-af12-7c0e04fe59a3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-DeleteServersTestJSON-volume-1735692310',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4e46ec4a-c69b-4a8d-b661-2d1bc20ed360,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8f92b50dc0364842980ce41af830ba1a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:02Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:02Z,use_quota=True,user_id='a2745d6923e347939e2c67015aefadda',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:02.225039 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (405f3caa-39e1-4f56-8471-7b801d299e2d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:02.229903 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (405f3caa-39e1-4f56-8471-7b801d299e2d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:02.235492 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3df1423d-6c3e-4810-bece-7ff4d49c7c9f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:02.239956 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3df1423d-6c3e-4810-bece-7ff4d49c7c9f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4e46ec4a-c69b-4a8d-b661-2d1bc20ed360', 'volume_name': 'volume-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:02.244931 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7ec9bb99-6188-448c-9bb6-bf03f201d7e0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:02.250076 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7ec9bb99-6188-448c-9bb6-bf03f201d7e0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:02.259674 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (94b2ec9e-b010-4193-a7c9-a69156031e20) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:02.260806 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Volume 4e46ec4a-c69b-4a8d-b661-2d1bc20ed360: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360', 'volume_size': 1} Oct 01 21:13:02.262002 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] creating volume 'volume-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:02.274791 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:02.384816 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:02.386683 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:02.390505 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (94b2ec9e-b010-4193-a7c9-a69156031e20) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:02.398321 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e4c14e62-9243-4dde-a3b5-d8ba52bc552e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:02.429596 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Volume volume-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360 (4e46ec4a-c69b-4a8d-b661-2d1bc20ed360): created successfully Oct 01 21:13:02.441947 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e4c14e62-9243-4dde-a3b5-d8ba52bc552e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:02.459095 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Flow 'volume_create_manager' (ef03c92b-40b1-4f77-9a6b-a2e290675941) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:02.502032 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-47c6e4d8-40fe-4e19-b958-d1bdf48ed2f2 tempest-DeleteServersTestJSON-611460547 None] Created volume successfully. Oct 01 21:13:02.537380 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] Acquiring lock "cinder-2c4d0b33-bffa-4227-94e9-5dcb58c177bf-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:02.626636 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] Lock "cinder-2c4d0b33-bffa-4227-94e9-5dcb58c177bf-delete_volume" acquired by "delete_volume" :: waited 0.082s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:02.657209 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] Created reservations ['ad50ed13-8e8a-4a03-9f64-a4c9be3afac9', '1ab0449a-01ac-4cc5-90d2-f7bd72282d0f', '983f6c56-563e-444f-adc0-7d549eeda9e6', '4520eb18-bba2-469e-b0a7-c57cb9c92907'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:02.692226 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:02.772664 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:02.777845 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] Volume volume-2c4d0b33-bffa-4227-94e9-5dcb58c177bf is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:02.793535 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] deleting rbd volume volume-2c4d0b33-bffa-4227-94e9-5dcb58c177bf {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:02.924205 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:03.061891 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:03.176126 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:03.188878 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] Deleted volume successfully. Oct 01 21:13:03.227979 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-76369717-4bc2-4c70-a5b4-80c1f7c8f7cd tempest-VolumesServicesTestJSON-1055956945 None] Lock "cinder-d91bc858-3e21-4ae4-9e00-a8b731207dbb-delete_volume" released by "delete_volume" :: held 1.606s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:03.291858 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] Created reservations ['15309377-6d06-4395-a8bf-54cf4093a9f3', '7737ebc9-1766-4d13-b495-fc1bf4980401', '42ed0890-4641-4ae2-9707-eeee4743c3d5', '5ff8cabf-40b9-47ef-a1b1-3c58d5edea11'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:03.376847 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=50,cinder:UPDATE=39,cinder:DELETE=5,cinder:INSERT=6 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:03.464450 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:03.522921 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:03.590033 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:03.602889 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] Deleted volume successfully. Oct 01 21:13:03.643576 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-63fce307-8eba-434d-8e38-66340300b095 tempest-VolumesActionsTest-679890519 None] Lock "cinder-2c4d0b33-bffa-4227-94e9-5dcb58c177bf-delete_volume" released by "delete_volume" :: held 1.024s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:04.546213 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:04.597366 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (2f78bea2-6994-436c-8f55-d1e710be5ed5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:04.612776 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b08ae0ef-e5cb-4b12-bbd4-b5c93bce8664) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:04.675988 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b08ae0ef-e5cb-4b12-bbd4-b5c93bce8664) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:04Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1007200204',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4688dd6b-a1cd-4ffe-a941-ba9b43c187fd,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='55864f798646443bbdfe30f8b9fbefb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:04Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:04Z,use_quota=True,user_id='78cc9dbf1b344a35b9be09ad32549a62',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:04.693453 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ff95e104-57c0-439b-a4e8-3a93b693b737) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:04.699972 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ff95e104-57c0-439b-a4e8-3a93b693b737) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:04.720218 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (18e27616-9398-4afc-b0e4-efecee7fa473) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:04.727435 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (18e27616-9398-4afc-b0e4-efecee7fa473) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4688dd6b-a1cd-4ffe-a941-ba9b43c187fd', 'volume_name': 'volume-4688dd6b-a1cd-4ffe-a941-ba9b43c187fd', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:04.740413 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (eaf54d64-4f12-4447-a28f-0a31007f222f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:04.750577 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (eaf54d64-4f12-4447-a28f-0a31007f222f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:04.761560 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4e1aab61-1092-47df-b9c8-cdb3977b1150) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:04.762778 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Volume 4688dd6b-a1cd-4ffe-a941-ba9b43c187fd: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4688dd6b-a1cd-4ffe-a941-ba9b43c187fd', 'volume_size': 1} Oct 01 21:13:04.763180 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] creating volume 'volume-4688dd6b-a1cd-4ffe-a941-ba9b43c187fd' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:04.766751 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:04.837601 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-f601e57f-ded9-41ce-87b5-46aed4c87cf4 req-c55eb9de-fc81-46c1-8308-1de76043373d tempest-DeleteServersTestJSON-611460547 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:13:04.923530 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:04.924394 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:04.933636 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4e1aab61-1092-47df-b9c8-cdb3977b1150) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4688dd6b-a1cd-4ffe-a941-ba9b43c187fd', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:04.946374 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6b1645d6-c123-4ac9-a32c-5b778d48d8d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:04.968784 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Volume volume-4688dd6b-a1cd-4ffe-a941-ba9b43c187fd (4688dd6b-a1cd-4ffe-a941-ba9b43c187fd): created successfully Oct 01 21:13:04.975375 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6b1645d6-c123-4ac9-a32c-5b778d48d8d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:04.982714 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (2f78bea2-6994-436c-8f55-d1e710be5ed5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:04.995005 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6a82424d-dc30-4996-8070-de6c15cf56e9 tempest-VolumesActionsTest-1835404292 None] Created volume successfully. Oct 01 21:13:05.947795 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] Acquiring lock "cinder-4688dd6b-a1cd-4ffe-a941-ba9b43c187fd-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:05.983422 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] Lock "cinder-4688dd6b-a1cd-4ffe-a941-ba9b43c187fd-delete_volume" acquired by "delete_volume" :: waited 0.036s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:06.068835 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:06.171215 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:06.173228 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] Volume volume-4688dd6b-a1cd-4ffe-a941-ba9b43c187fd is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:06.184039 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] deleting rbd volume volume-4688dd6b-a1cd-4ffe-a941-ba9b43c187fd {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:06.400758 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-f601e57f-ded9-41ce-87b5-46aed4c87cf4 req-c55eb9de-fc81-46c1-8308-1de76043373d tempest-DeleteServersTestJSON-611460547 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.560s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:13:06.401711 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-f601e57f-ded9-41ce-87b5-46aed4c87cf4 req-c55eb9de-fc81-46c1-8308-1de76043373d tempest-DeleteServersTestJSON-611460547 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '4e46ec4a-c69b-4a8d-b661-2d1bc20ed360', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:13:06.462984 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-f601e57f-ded9-41ce-87b5-46aed4c87cf4 req-c55eb9de-fc81-46c1-8308-1de76043373d tempest-DeleteServersTestJSON-611460547 None] Connection info returned from driver {'name': 'volumes/volume-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '4e46ec4a-c69b-4a8d-b661-2d1bc20ed360', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '57c5e934-127e-4ec6-adb9-cd26bc8112b9'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:13:06.664503 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=64,cinder:UPDATE=26,cinder:DELETE=3,cinder:INSERT=7 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:06.670321 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] Created reservations ['3197c8b1-c773-4430-b661-7f8ecad4c10e', '2f59d03c-c18b-4524-b795-7b710047c9f7', '7696496f-f4ed-45a9-99df-a45779e1d4e2', '8a5f7dd3-5b2e-4fd4-9950-fd158703cf17'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:06.693522 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-f601e57f-ded9-41ce-87b5-46aed4c87cf4 req-c55eb9de-fc81-46c1-8308-1de76043373d tempest-DeleteServersTestJSON-611460547 None] attachment_update completed successfully. Oct 01 21:13:06.742264 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:06.834251 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:06.917940 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:06.923122 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] Deleted volume successfully. Oct 01 21:13:06.949929 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fbc05e7c-7136-4d9d-9689-e0151daea230 tempest-VolumesActionsTest-679890519 None] Lock "cinder-4688dd6b-a1cd-4ffe-a941-ba9b43c187fd-delete_volume" released by "delete_volume" :: held 0.967s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:07.985742 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:08.025646 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (71039c7f-2fc9-4d30-a759-1d5fca9a9e69) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:08.042628 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5635280c-86ea-4ceb-9f38-663458bab06a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:08.063869 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5635280c-86ea-4ceb-9f38-663458bab06a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1235157504',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=108f53fd-07ac-413e-9c8c-46e9407e786f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='55864f798646443bbdfe30f8b9fbefb1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:08Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:08Z,use_quota=True,user_id='78cc9dbf1b344a35b9be09ad32549a62',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:08.073488 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (98986924-21e5-4fe7-90c1-24f1274d005f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:08.078032 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (98986924-21e5-4fe7-90c1-24f1274d005f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:08.083326 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (008a9740-923e-4608-8fd4-4d19c5a78732) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:08.087627 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (008a9740-923e-4608-8fd4-4d19c5a78732) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '108f53fd-07ac-413e-9c8c-46e9407e786f', 'volume_name': 'volume-108f53fd-07ac-413e-9c8c-46e9407e786f', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:08.093708 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1ac08a23-75ea-4139-82c2-e8907fc1a649) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:08.097887 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1ac08a23-75ea-4139-82c2-e8907fc1a649) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:08.111667 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d5056b95-0151-4d0e-bfaf-abe2ca18474c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:08.112746 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Volume 108f53fd-07ac-413e-9c8c-46e9407e786f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-108f53fd-07ac-413e-9c8c-46e9407e786f', 'volume_size': 1} Oct 01 21:13:08.113156 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] creating volume 'volume-108f53fd-07ac-413e-9c8c-46e9407e786f' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:08.115199 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:08.241871 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:08.242625 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:08.248451 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d5056b95-0151-4d0e-bfaf-abe2ca18474c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-108f53fd-07ac-413e-9c8c-46e9407e786f', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:08.261904 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (812c08fa-2934-446c-977a-c45d999274b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:08.286327 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Volume volume-108f53fd-07ac-413e-9c8c-46e9407e786f (108f53fd-07ac-413e-9c8c-46e9407e786f): created successfully Oct 01 21:13:08.291348 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (812c08fa-2934-446c-977a-c45d999274b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:08.302681 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Flow 'volume_create_manager' (71039c7f-2fc9-4d30-a759-1d5fca9a9e69) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:08.322717 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-df4cbd49-d201-46b4-b02e-f667aff12cfb tempest-VolumesActionsTest-1835404292 None] Created volume successfully. Oct 01 21:13:10.186399 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] Acquiring lock "cinder-108f53fd-07ac-413e-9c8c-46e9407e786f-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:10.208739 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] Lock "cinder-108f53fd-07ac-413e-9c8c-46e9407e786f-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:10.352634 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:10.451737 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:10.452177 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] Volume volume-108f53fd-07ac-413e-9c8c-46e9407e786f is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:10.465366 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] deleting rbd volume volume-108f53fd-07ac-413e-9c8c-46e9407e786f {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:10.984276 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] Created reservations ['2c03f2e9-b1d6-4855-bd5b-195125bafaf7', '4acc50ca-5165-4399-aa30-d27ffe959497', '24d65e1d-f600-4b4a-8dc4-00351d403382', '6a17eeae-ac5b-43d8-97b6-999e77b05842'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:11.120515 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:11.214626 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:11.325927 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:11.334741 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] Deleted volume successfully. Oct 01 21:13:11.359788 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-37face43-c10d-478e-8be2-3ece4fc59496 tempest-VolumesActionsTest-1835404292 None] Lock "cinder-108f53fd-07ac-413e-9c8c-46e9407e786f-delete_volume" released by "delete_volume" :: held 1.151s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:11.654412 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] Acquiring lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:11.667714 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] Lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:11.730811 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:11.788659 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:11.788659 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] Volume volume-642746eb-e0e8-4aed-8b8d-67e3230dbcb8 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:11.799581 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] deleting rbd volume volume-642746eb-e0e8-4aed-8b8d-67e3230dbcb8 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:12.180188 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] Created reservations ['800d0958-f307-49f5-be03-f98a826d5b56', 'e5db03e0-ad1b-4c69-967e-2eb4ce2d2e14', '60686bfd-32e4-4906-b24f-cb983b4d0b46', '57b70def-a385-4392-9407-794f19f0745b'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:12.301498 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:12.371526 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:12.458025 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:12.463937 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] Deleted volume successfully. Oct 01 21:13:12.488496 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-695f56c5-a3bc-4012-92d4-90af165c567b tempest-VolumesActionsTest-1835404292 None] Lock "cinder-642746eb-e0e8-4aed-8b8d-67e3230dbcb8-delete_volume" released by "delete_volume" :: held 0.819s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:13.020128 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:13.062755 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Flow 'volume_create_manager' (37c77395-be13-4305-b8ae-ace9d738c12f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:13.079888 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a3cb36d7-ccc8-4e69-a688-8aa4ec1c8d21) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:13.150809 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a3cb36d7-ccc8-4e69-a688-8aa4ec1c8d21) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:13Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeTypesAccessTest-Volume-399564483',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='fdc98fc841664be2bc220da1a2f0a9fe',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:13Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:13Z,use_quota=True,user_id='f8d10af82ef249feb8bfb058a6633730',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(d2d81674-f01b-4469-9de5-a6d77f6dd1a4),volume_type_id=d2d81674-f01b-4469-9de5-a6d77f6dd1a4)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:13.175131 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e1b1f6bc-6969-404c-97a2-a8525117f9bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:13.179473 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e1b1f6bc-6969-404c-97a2-a8525117f9bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:13.194844 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c0dbe928-fef1-4a55-8443-9a9ecc7776af) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:13.199698 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c0dbe928-fef1-4a55-8443-9a9ecc7776af) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35', 'volume_name': 'volume-ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:13.217235 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (70a49ffc-f643-428b-a7f1-d45aa8032db5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:13.224238 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (70a49ffc-f643-428b-a7f1-d45aa8032db5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:13.233628 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cd001074-64e2-43f3-9d1e-aa4aadd6d114) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:13.234927 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Volume ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35', 'volume_size': 1} Oct 01 21:13:13.235326 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] creating volume 'volume-ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:13.236328 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:13.340471 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:13.341146 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:13.347351 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cd001074-64e2-43f3-9d1e-aa4aadd6d114) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:13.352012 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c26d1437-5c28-4bff-91e2-de7083e0a167) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:13.365677 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Volume volume-ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35 (ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35): created successfully Oct 01 21:13:13.369695 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c26d1437-5c28-4bff-91e2-de7083e0a167) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:13.376862 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Flow 'volume_create_manager' (37c77395-be13-4305-b8ae-ace9d738c12f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:13.390002 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-80f0dde2-4e6c-42a7-92fd-3200d89ba0c9 tempest-VolumeTypesAccessTest-2025261266 None] Created volume successfully. Oct 01 21:13:13.402472 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=56,cinder:UPDATE=36,cinder:DELETE=3,cinder:INSERT=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:14.421853 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-f16eeeab-67a5-4ce2-b352-c6245906dbfb req-011252fe-27b0-4e07-8707-3f0d8d55ba97 tempest-DeleteServersTestJSON-611460547 None] Terminate volume connection completed successfully. Oct 01 21:13:14.422577 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-f16eeeab-67a5-4ce2-b352-c6245906dbfb req-011252fe-27b0-4e07-8707-3f0d8d55ba97 tempest-DeleteServersTestJSON-611460547 None] Deleting attachment 57c5e934-127e-4ec6-adb9-cd26bc8112b9. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:13:14.804466 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] Acquiring lock "cinder-ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:14.827494 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] Lock "cinder-ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35-delete_volume" acquired by "delete_volume" :: waited 0.024s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:14.992746 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:15.086330 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:15.086872 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] Volume volume-ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:15.093145 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] deleting rbd volume volume-ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:15.478961 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-671259643 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-671259643, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:13:15.480964 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-671259643 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-671259643, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:13:15.508749 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] Created reservations ['8d6e6924-9362-411a-8bbc-d5980f421b56', 'eebdd74f-0c5d-410c-bfe0-460cbce99792', 'faa20ceb-8ae5-4823-9ebc-f235308aac49', '42aaebc8-ea9d-45ce-89f5-41fe36ab4a4e'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:15.712920 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:15.820934 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:15.890983 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:15.915237 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] Deleted volume successfully. Oct 01 21:13:15.955412 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0a2e3b94-43b4-45c5-ae8d-bb7d4fbebeb6 tempest-VolumeTypesAccessTest-2025261266 None] Lock "cinder-ff87f6c8-1af6-4dc4-9ac4-c9eeb6d98e35-delete_volume" released by "delete_volume" :: held 1.128s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:16.642327 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:16.692979 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Flow 'volume_create_manager' (75e82451-f807-4e00-8de6-112e1a013934) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:16.702746 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5fa5d1c2-5683-4f6d-8074-b71441e35235) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:16.723859 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5fa5d1c2-5683-4f6d-8074-b71441e35235) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-633504831',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=47e1224d-e4a1-47ed-a3ce-0f9d5b65a943,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='588321452fd442ee8b78e7d7e4fa4638',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:17Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:17Z,use_quota=True,user_id='20dc4cd847944bf09273b91c1cf6952e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:16.734185 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (733da3cd-26f7-4e34-b60f-1d0570ff1cda) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:16.739438 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (733da3cd-26f7-4e34-b60f-1d0570ff1cda) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:16.746357 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (eb4d763e-2cfd-4645-87d3-677100bdfb69) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:16.752076 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (eb4d763e-2cfd-4645-87d3-677100bdfb69) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '47e1224d-e4a1-47ed-a3ce-0f9d5b65a943', 'volume_name': 'volume-47e1224d-e4a1-47ed-a3ce-0f9d5b65a943', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:16.758468 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c464596c-b5a4-420d-af1c-a222a23f63ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:16.764349 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c464596c-b5a4-420d-af1c-a222a23f63ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:16.769953 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d7766f4b-6d85-4cb1-a07e-bd0f7e95de71) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:16.771320 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Volume 47e1224d-e4a1-47ed-a3ce-0f9d5b65a943: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-47e1224d-e4a1-47ed-a3ce-0f9d5b65a943', 'volume_size': 1} Oct 01 21:13:16.771922 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] creating volume 'volume-47e1224d-e4a1-47ed-a3ce-0f9d5b65a943' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:16.772879 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:16.917858 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:16.918430 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:16.923128 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d7766f4b-6d85-4cb1-a07e-bd0f7e95de71) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-47e1224d-e4a1-47ed-a3ce-0f9d5b65a943', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:16.928260 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d555deb0-d650-42d0-91f9-a2eaf5d52f0c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:16.939980 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=33,cinder:SELECT=58,cinder:DELETE=3,cinder:INSERT=6 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:16.958534 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Volume volume-47e1224d-e4a1-47ed-a3ce-0f9d5b65a943 (47e1224d-e4a1-47ed-a3ce-0f9d5b65a943): created successfully Oct 01 21:13:16.969534 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d555deb0-d650-42d0-91f9-a2eaf5d52f0c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:16.976741 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Flow 'volume_create_manager' (75e82451-f807-4e00-8de6-112e1a013934) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:16.993048 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e3c390f5-1386-4e41-8ec8-78ab8f8e16c0 tempest-VolumesTestJSON-1405758743 None] Created volume successfully. Oct 01 21:13:19.164859 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:19.221559 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Flow 'volume_create_manager' (ef0e944b-dd22-468f-ad29-bfa075c07b94) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:19.227993 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (529a62c4-e560-495e-a791-9927dd8f4b68) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:19.262543 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (529a62c4-e560-495e-a791-9927dd8f4b68) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-458526643',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=134a663b-2510-49ba-b326-77540a1d42ca,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='588321452fd442ee8b78e7d7e4fa4638',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:19Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:19Z,use_quota=True,user_id='20dc4cd847944bf09273b91c1cf6952e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:19.269173 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e5d10cff-9743-4e19-9d8d-177bb4039f73) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:19.276900 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e5d10cff-9743-4e19-9d8d-177bb4039f73) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:19.283080 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5e2e12eb-575c-45d0-8100-d69cff081c5d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:19.288130 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5e2e12eb-575c-45d0-8100-d69cff081c5d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '134a663b-2510-49ba-b326-77540a1d42ca', 'volume_name': 'volume-134a663b-2510-49ba-b326-77540a1d42ca', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:19.295578 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (39d20080-0ec4-4315-bb0d-718afb794e62) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:19.301880 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (39d20080-0ec4-4315-bb0d-718afb794e62) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:19.310353 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d4e0a028-c2d7-485b-8a4b-86f87383b079) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:19.312682 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Volume 134a663b-2510-49ba-b326-77540a1d42ca: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-134a663b-2510-49ba-b326-77540a1d42ca', 'volume_size': 1} Oct 01 21:13:19.313861 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] creating volume 'volume-134a663b-2510-49ba-b326-77540a1d42ca' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:19.314851 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:19.401362 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:19.402101 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:19.406686 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d4e0a028-c2d7-485b-8a4b-86f87383b079) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-134a663b-2510-49ba-b326-77540a1d42ca', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:19.412161 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f637ff23-4293-4668-9a22-48a2f93f74e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:19.424711 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Volume volume-134a663b-2510-49ba-b326-77540a1d42ca (134a663b-2510-49ba-b326-77540a1d42ca): created successfully Oct 01 21:13:19.429286 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f637ff23-4293-4668-9a22-48a2f93f74e1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:19.440653 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Flow 'volume_create_manager' (ef0e944b-dd22-468f-ad29-bfa075c07b94) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:19.455294 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b8352446-176d-459e-8345-fd45a9d2bccb tempest-VolumesTestJSON-1405758743 None] Created volume successfully. Oct 01 21:13:20.680335 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:20.704091 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Flow 'volume_create_manager' (a5068c4a-0fc7-458d-a9aa-37171adc2d0f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:20.712447 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4e1eedb0-89f1-4cd7-b1ee-d22a5d1686b5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:20.730855 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4e1eedb0-89f1-4cd7-b1ee-d22a5d1686b5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-1453731444',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3e590a02-bca6-49dd-b076-2db79b6971cb,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='588321452fd442ee8b78e7d7e4fa4638',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:21Z,use_quota=True,user_id='20dc4cd847944bf09273b91c1cf6952e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:20.736907 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6597b6a5-011e-4d66-9de8-f688dc6620aa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:20.741268 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6597b6a5-011e-4d66-9de8-f688dc6620aa) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:20.746279 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (420ef807-737d-4b62-9b04-9389f91998b0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:20.750254 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (420ef807-737d-4b62-9b04-9389f91998b0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '3e590a02-bca6-49dd-b076-2db79b6971cb', 'volume_name': 'volume-3e590a02-bca6-49dd-b076-2db79b6971cb', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:20.755308 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (124ec713-35e8-4d55-9433-e878316c10d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:20.759050 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (124ec713-35e8-4d55-9433-e878316c10d7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:20.764550 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3a9a9bc3-2a5b-4493-83a2-a80e4db639ea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:20.765958 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Volume 3e590a02-bca6-49dd-b076-2db79b6971cb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-3e590a02-bca6-49dd-b076-2db79b6971cb', 'volume_size': 1} Oct 01 21:13:20.766767 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] creating volume 'volume-3e590a02-bca6-49dd-b076-2db79b6971cb' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:20.767432 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:20.882181 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:20.883026 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:20.888268 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3a9a9bc3-2a5b-4493-83a2-a80e4db639ea) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3e590a02-bca6-49dd-b076-2db79b6971cb', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:20.895214 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a85c61ae-7b57-4ccf-8983-924bef1415e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:20.934585 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Volume volume-3e590a02-bca6-49dd-b076-2db79b6971cb (3e590a02-bca6-49dd-b076-2db79b6971cb): created successfully Oct 01 21:13:20.945800 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a85c61ae-7b57-4ccf-8983-924bef1415e5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:20.971751 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Flow 'volume_create_manager' (a5068c4a-0fc7-458d-a9aa-37171adc2d0f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:20.992319 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-49f0d922-45a3-4ead-ab7d-bddcb777f68d tempest-VolumesTestJSON-1405758743 None] Created volume successfully. Oct 01 21:13:23.806729 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:13:23.808883 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:23.878599 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:23.954725 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:24.815047 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] Acquiring lock "cinder-3e590a02-bca6-49dd-b076-2db79b6971cb-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:24.832903 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] Lock "cinder-3e590a02-bca6-49dd-b076-2db79b6971cb-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:24.886814 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:24.975270 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:24.976038 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] Volume volume-3e590a02-bca6-49dd-b076-2db79b6971cb is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:24.991048 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] deleting rbd volume volume-3e590a02-bca6-49dd-b076-2db79b6971cb {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:25.194110 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:25.224119 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Flow 'volume_create_manager' (0294a43c-6e9c-44dc-8423-d186b67d5f09) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:25.234103 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (61aec652-0269-4cf6-a1c1-1c342483dbca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:25.263315 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (61aec652-0269-4cf6-a1c1-1c342483dbca) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:25Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotMetadataTestJSON-Volume-1404626546',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cce3817e-6f78-497c-a696-284a3737ca26,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d2f356873b84cca908a01cc30d7cac0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:25Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:25Z,use_quota=True,user_id='9399e9638e464708bc007502e2f571a2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:25.269278 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fe070b40-9e95-408d-9fe7-c070818e9740) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:25.273504 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fe070b40-9e95-408d-9fe7-c070818e9740) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:25.280835 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (46bec3cb-ccb4-4dfb-af8f-8170cf298a06) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:25.285527 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (46bec3cb-ccb4-4dfb-af8f-8170cf298a06) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'cce3817e-6f78-497c-a696-284a3737ca26', 'volume_name': 'volume-cce3817e-6f78-497c-a696-284a3737ca26', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:25.297912 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (60a3cdb6-306d-46ce-9539-6fd014041d62) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:25.302986 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (60a3cdb6-306d-46ce-9539-6fd014041d62) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:25.308156 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (067c7bb2-0ad5-45ff-9371-5b8701c6fb43) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:25.309241 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Volume cce3817e-6f78-497c-a696-284a3737ca26: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-cce3817e-6f78-497c-a696-284a3737ca26', 'volume_size': 1} Oct 01 21:13:25.309711 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] creating volume 'volume-cce3817e-6f78-497c-a696-284a3737ca26' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:25.310295 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:25.338756 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-b3f34693-17be-400d-b8fa-fd2085d89bae req-c3a30c99-ce1b-43b7-b8e6-5d8674b15cd5 tempest-AttachVolumeShelveTestJSON-705121323 None] No connector for attachment c852fc31-92b8-492c-9def-c0134774875c; skipping storage backend terminate_connection call. {{(pid=133391) _connection_terminate /opt/stack/cinder/cinder/volume/manager.py:4953}} Oct 01 21:13:25.339823 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-b3f34693-17be-400d-b8fa-fd2085d89bae req-c3a30c99-ce1b-43b7-b8e6-5d8674b15cd5 tempest-AttachVolumeShelveTestJSON-705121323 None] Deleting attachment c852fc31-92b8-492c-9def-c0134774875c. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:13:25.403906 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:25.404441 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:25.408515 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (067c7bb2-0ad5-45ff-9371-5b8701c6fb43) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cce3817e-6f78-497c-a696-284a3737ca26', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:25.414143 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bfe47489-46be-4ba2-bd69-bcf9ea384c91) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:25.434595 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Volume volume-cce3817e-6f78-497c-a696-284a3737ca26 (cce3817e-6f78-497c-a696-284a3737ca26): created successfully Oct 01 21:13:25.438834 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bfe47489-46be-4ba2-bd69-bcf9ea384c91) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:25.445018 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Flow 'volume_create_manager' (0294a43c-6e9c-44dc-8423-d186b67d5f09) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:25.467474 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f3c75864-fd02-447c-8845-9910a94f882a tempest-SnapshotMetadataTestJSON-462018219 None] Created volume successfully. Oct 01 21:13:25.473955 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] Created reservations ['86e5e2b1-293a-4be7-b1dd-c5138bf743bb', '126456e7-46b6-442c-a128-891084dd6928', '819cff75-4c2e-4393-8ed5-459f90a6df5e', '1ea47740-3ef8-4881-b3a3-bdb9b078ebcc'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:25.564907 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:25.621515 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:25.667527 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:25.672167 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] Deleted volume successfully. Oct 01 21:13:25.687553 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f31c8e7c-c409-4661-81fc-edc79dde6060 tempest-VolumesTestJSON-1405758743 None] Lock "cinder-3e590a02-bca6-49dd-b076-2db79b6971cb-delete_volume" released by "delete_volume" :: held 0.856s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:26.206908 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] Acquiring lock "cinder-134a663b-2510-49ba-b326-77540a1d42ca-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:26.225338 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] Lock "cinder-134a663b-2510-49ba-b326-77540a1d42ca-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:26.303346 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:26.357663 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:26.358177 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] Volume volume-134a663b-2510-49ba-b326-77540a1d42ca is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:26.365517 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] deleting rbd volume volume-134a663b-2510-49ba-b326-77540a1d42ca {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:26.610921 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-67f46a31-3138-49e9-bec1-b06394782c9f tempest-SnapshotMetadataTestJSON-462018219 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:26.694831 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] Created reservations ['3ca4ddf2-25be-4fe5-be6f-3b239a4aca33', 'da1a6522-2949-4c03-8941-75aacd10468b', '4f1ed92a-4533-4a00-87f1-6f2b5f706217', '27db8571-e679-4405-a4c3-e449d37e53dd'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:26.744951 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:26.781085 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:26.824076 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:26.829685 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] Deleted volume successfully. Oct 01 21:13:26.844207 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9cfb2dad-8f61-48bb-8968-22b850e0a7b1 tempest-VolumesTestJSON-1405758743 None] Lock "cinder-134a663b-2510-49ba-b326-77540a1d42ca-delete_volume" released by "delete_volume" :: held 0.619s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:27.274259 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-67f46a31-3138-49e9-bec1-b06394782c9f tempest-SnapshotMetadataTestJSON-462018219 None] Create snapshot completed successfully Oct 01 21:13:27.277751 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=55,cinder:UPDATE=34,cinder:INSERT=7,cinder:DELETE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:27.395669 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] Acquiring lock "cinder-47e1224d-e4a1-47ed-a3ce-0f9d5b65a943-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:27.411505 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] Lock "cinder-47e1224d-e4a1-47ed-a3ce-0f9d5b65a943-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:27.459476 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:27.511887 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:27.512311 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] Volume volume-47e1224d-e4a1-47ed-a3ce-0f9d5b65a943 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:27.520358 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] deleting rbd volume volume-47e1224d-e4a1-47ed-a3ce-0f9d5b65a943 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:27.789349 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] Created reservations ['2709cc97-1ed0-44c0-981d-0d345ef0fcc4', '4a5a1f5e-295d-4cf1-acf4-48178d631df8', 'fab75aca-c038-43f4-8576-510f0f7c17d8', '18aec698-465f-44ee-b95a-cb72285a6328'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:27.871124 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:27.921607 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:27.965797 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:27.977080 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] Deleted volume successfully. Oct 01 21:13:27.999827 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5918a406-14e4-45b9-9d1b-653d32e2a719 tempest-VolumesTestJSON-1405758743 None] Lock "cinder-47e1224d-e4a1-47ed-a3ce-0f9d5b65a943-delete_volume" released by "delete_volume" :: held 0.585s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:28.403991 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-55512c70-486f-4f7e-8419-bf0f6d97d596 tempest-SnapshotMetadataTestJSON-462018219 None] Acquiring lock "cinder-50ce40c5-0605-47e9-ae1b-20e62d621c95-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:28.419791 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-55512c70-486f-4f7e-8419-bf0f6d97d596 tempest-SnapshotMetadataTestJSON-462018219 None] Lock "cinder-50ce40c5-0605-47e9-ae1b-20e62d621c95-delete_snapshot" acquired by "delete_snapshot" :: waited 0.016s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:28.421332 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-55512c70-486f-4f7e-8419-bf0f6d97d596 tempest-SnapshotMetadataTestJSON-462018219 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:28.923305 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:28.946163 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Flow 'volume_create_manager' (ed8ac73a-3316-4e0a-a2c0-1cd3cc6631dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:28.953853 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e87c10b0-0dc9-4ca9-b789-3369e7ff9533) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:28.981854 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e87c10b0-0dc9-4ca9-b789-3369e7ff9533) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-2007088825',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=20b8155f-8af6-40fb-aa4d-713df3920827,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c6b627e68c7b42c18cb91b2e8308b3a6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:29Z,use_quota=True,user_id='f542f3958542490a811c4dd8fe8fe04d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:28.987570 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (112a36d0-08b7-42ac-9062-78947178587d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:28.991742 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (112a36d0-08b7-42ac-9062-78947178587d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:28.997101 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0e761f40-45e8-48b2-8605-08e6e6b0d374) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:29.001269 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0e761f40-45e8-48b2-8605-08e6e6b0d374) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '20b8155f-8af6-40fb-aa4d-713df3920827', 'volume_name': 'volume-20b8155f-8af6-40fb-aa4d-713df3920827', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:29.006672 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (de558423-9918-4105-a944-c685f53099ea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:29.011483 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (de558423-9918-4105-a944-c685f53099ea) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:29.017102 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (40a994ea-0cab-4f50-a425-bdf758282758) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:29.018301 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Volume 20b8155f-8af6-40fb-aa4d-713df3920827: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-20b8155f-8af6-40fb-aa4d-713df3920827', 'volume_size': 1} Oct 01 21:13:29.018850 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] creating volume 'volume-20b8155f-8af6-40fb-aa4d-713df3920827' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:29.019887 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:29.115748 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:29.116292 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:29.122012 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (40a994ea-0cab-4f50-a425-bdf758282758) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-20b8155f-8af6-40fb-aa4d-713df3920827', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:29.128818 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0f6f7cd2-4477-49c8-b9e1-44f82c5e585b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:29.144455 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Volume volume-20b8155f-8af6-40fb-aa4d-713df3920827 (20b8155f-8af6-40fb-aa4d-713df3920827): created successfully Oct 01 21:13:29.149046 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0f6f7cd2-4477-49c8-b9e1-44f82c5e585b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:29.154583 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Flow 'volume_create_manager' (ed8ac73a-3316-4e0a-a2c0-1cd3cc6631dd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:29.166677 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3b6e4569-6253-49cd-9607-bec2620f5572 tempest-VolumesMetadataTest-2085721061 None] Created volume successfully. Oct 01 21:13:29.259586 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=67,cinder:DELETE=5,cinder:UPDATE=23,cinder:INSERT=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:29.318018 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-55512c70-486f-4f7e-8419-bf0f6d97d596 tempest-SnapshotMetadataTestJSON-462018219 None] Created reservations ['0e695094-7fa7-41b7-9472-264564f9595b', '93e7adc4-ad03-4f78-acb9-0619bb1a2614', 'a7dbb69e-feff-4c9f-b168-ada642abe36e', '6d6249a5-612f-455f-8895-a7f4ec8f3480'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:29.389869 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-55512c70-486f-4f7e-8419-bf0f6d97d596 tempest-SnapshotMetadataTestJSON-462018219 None] Delete snapshot completed successfully. Oct 01 21:13:29.402094 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-55512c70-486f-4f7e-8419-bf0f6d97d596 tempest-SnapshotMetadataTestJSON-462018219 None] Lock "cinder-50ce40c5-0605-47e9-ae1b-20e62d621c95-delete_snapshot" released by "delete_snapshot" :: held 0.982s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:29.619325 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] Acquiring lock "cinder-cce3817e-6f78-497c-a696-284a3737ca26-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:29.633345 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] Lock "cinder-cce3817e-6f78-497c-a696-284a3737ca26-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:29.705558 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:29.834294 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:29.835099 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] Volume volume-cce3817e-6f78-497c-a696-284a3737ca26 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:29.848314 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] deleting rbd volume volume-cce3817e-6f78-497c-a696-284a3737ca26 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:30.091356 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:30.142610 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Flow 'volume_create_manager' (77f3ce02-7433-400f-8c39-ab48d5829615) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:30.155371 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (952858be-4e82-4a08-9689-40f96b2e9e20) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:30.190555 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (952858be-4e82-4a08-9689-40f96b2e9e20) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-1711106703',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=46afd2fc-5506-4c1c-8c97-203abd3a41d9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='24a06f5da36b4488a1b8ea01729b0e6c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:30Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:30Z,use_quota=True,user_id='902ba44d318d4a01a510f732d27467c9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:30.197017 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fc87f959-f956-4611-9b09-12c614ecf07b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:30.200904 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fc87f959-f956-4611-9b09-12c614ecf07b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:30.206355 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (05e07cfb-b986-473e-ab8d-32b36a2adb00) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:30.228164 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (05e07cfb-b986-473e-ab8d-32b36a2adb00) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '46afd2fc-5506-4c1c-8c97-203abd3a41d9', 'volume_name': 'volume-46afd2fc-5506-4c1c-8c97-203abd3a41d9', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:30.234018 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f79bce99-1d86-49fc-8618-4566aea4bae1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:30.238040 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f79bce99-1d86-49fc-8618-4566aea4bae1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:30.243550 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (81f8e3f1-aba0-4c34-8ccb-86d28b23fb64) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:30.244440 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Volume 46afd2fc-5506-4c1c-8c97-203abd3a41d9: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-46afd2fc-5506-4c1c-8c97-203abd3a41d9', 'volume_size': 1} Oct 01 21:13:30.244848 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] creating volume 'volume-46afd2fc-5506-4c1c-8c97-203abd3a41d9' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:30.245634 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:30.375221 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:30.376559 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:30.394294 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (81f8e3f1-aba0-4c34-8ccb-86d28b23fb64) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-46afd2fc-5506-4c1c-8c97-203abd3a41d9', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:30.405835 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (468efbcf-61cf-4f11-ae29-0af99e696e09) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:30.414287 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] Created reservations ['0ee11dd2-99d2-47e0-b92b-c897172a5d9d', '7a87aab8-5dbc-42f0-b5b1-f27e09ec762c', '9498e355-4bfc-445a-a80a-ca66a52ab691', 'ed2dfb65-86c9-4881-afac-7c2cde38aac3'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:30.423823 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Volume volume-46afd2fc-5506-4c1c-8c97-203abd3a41d9 (46afd2fc-5506-4c1c-8c97-203abd3a41d9): created successfully Oct 01 21:13:30.436049 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (468efbcf-61cf-4f11-ae29-0af99e696e09) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:30.450887 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Flow 'volume_create_manager' (77f3ce02-7433-400f-8c39-ab48d5829615) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:30.487142 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b598c2e6-c7fb-427f-840e-096394f5359b tempest-AttachVolumeNegativeTest-855754723 None] Created volume successfully. Oct 01 21:13:30.571586 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:30.663983 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:30.734461 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:30.738541 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] Deleted volume successfully. Oct 01 21:13:30.752895 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-074f030c-b277-4d72-bb55-105133c7a4fb tempest-SnapshotMetadataTestJSON-462018219 None] Lock "cinder-cce3817e-6f78-497c-a696-284a3737ca26-delete_volume" released by "delete_volume" :: held 1.120s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:31.349369 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] Acquiring lock "cinder-20b8155f-8af6-40fb-aa4d-713df3920827-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:31.377998 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] Lock "cinder-20b8155f-8af6-40fb-aa4d-713df3920827-delete_volume" acquired by "delete_volume" :: waited 0.029s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:31.459513 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:31.564540 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:31.565273 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] Volume volume-20b8155f-8af6-40fb-aa4d-713df3920827 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:31.576687 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] deleting rbd volume volume-20b8155f-8af6-40fb-aa4d-713df3920827 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:31.983614 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-9b9ad51e-4d69-40ea-af8f-f148298159eb req-553931dd-6caf-4fd2-8ab5-2095f73cd75a tempest-AttachVolumeNegativeTest-855754723 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:13:32.035892 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] Created reservations ['b267bef7-76cd-4b09-85a5-20210ce86e85', '83beb7a1-17bf-4b71-b4a6-e005b1a4f450', '0f2eaeb5-2995-468e-8f33-ab337f360b9f', '31ecc2f2-f2f6-4b35-acfc-a40056f87187'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:32.049422 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=58,cinder:INSERT=10,cinder:UPDATE=31,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:32.203230 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:32.286980 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:32.344208 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:32.349695 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] Deleted volume successfully. Oct 01 21:13:32.368253 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a6ac1136-b78d-40fa-9f0a-4e16b95ba798 tempest-VolumesMetadataTest-2085721061 None] Lock "cinder-20b8155f-8af6-40fb-aa4d-713df3920827-delete_volume" released by "delete_volume" :: held 0.990s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:33.188484 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-9b9ad51e-4d69-40ea-af8f-f148298159eb req-553931dd-6caf-4fd2-8ab5-2095f73cd75a tempest-AttachVolumeNegativeTest-855754723 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.206s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:13:33.189640 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-9b9ad51e-4d69-40ea-af8f-f148298159eb req-553931dd-6caf-4fd2-8ab5-2095f73cd75a tempest-AttachVolumeNegativeTest-855754723 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-46afd2fc-5506-4c1c-8c97-203abd3a41d9', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '46afd2fc-5506-4c1c-8c97-203abd3a41d9', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:13:33.241625 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-9b9ad51e-4d69-40ea-af8f-f148298159eb req-553931dd-6caf-4fd2-8ab5-2095f73cd75a tempest-AttachVolumeNegativeTest-855754723 None] Connection info returned from driver {'name': 'volumes/volume-46afd2fc-5506-4c1c-8c97-203abd3a41d9', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '46afd2fc-5506-4c1c-8c97-203abd3a41d9', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '95a95e19-cfbc-4f61-a817-f9547fe81247'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:13:33.368149 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-9b9ad51e-4d69-40ea-af8f-f148298159eb req-553931dd-6caf-4fd2-8ab5-2095f73cd75a tempest-AttachVolumeNegativeTest-855754723 None] attachment_update completed successfully. Oct 01 21:13:36.539619 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:36.595390 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Flow 'volume_create_manager' (b04e53db-5eb1-4235-9b43-194dac78bdde) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:36.603906 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (dfe6ed19-e8af-4df6-ab19-7594976227db) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:36.768807 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (dfe6ed19-e8af-4df6-ab19-7594976227db) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:36Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesDeleteCascade-Volume-1977771831',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=301a11d9-13b7-459f-90cd-f479e483c213,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='424cbcca79d947f7a629d36a2cc90847',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:36Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:36Z,use_quota=True,user_id='6e8b32fe37d342c2911d6e1ede9f37ee',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:36.778726 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6ffcc38d-fee3-4e21-aeb0-7762827e4f2c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:36.788530 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6ffcc38d-fee3-4e21-aeb0-7762827e4f2c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:36.802821 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1d314b92-6eb1-466c-9c30-d06876308cc0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:36.812705 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1d314b92-6eb1-466c-9c30-d06876308cc0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '301a11d9-13b7-459f-90cd-f479e483c213', 'volume_name': 'volume-301a11d9-13b7-459f-90cd-f479e483c213', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:36.820514 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0d2cb00c-7677-41bb-a385-865e0a58990d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:36.829491 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0d2cb00c-7677-41bb-a385-865e0a58990d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:36.839346 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (faff38d7-2cb5-4200-8195-f4fdd029c171) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:36.843096 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Volume 301a11d9-13b7-459f-90cd-f479e483c213: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-301a11d9-13b7-459f-90cd-f479e483c213', 'volume_size': 1} Oct 01 21:13:36.844525 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] creating volume 'volume-301a11d9-13b7-459f-90cd-f479e483c213' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:36.846131 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:37.019056 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:37.019774 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:37.036600 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (faff38d7-2cb5-4200-8195-f4fdd029c171) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-301a11d9-13b7-459f-90cd-f479e483c213', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:37.048638 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bd382eb4-badc-441a-b905-04591035134e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:37.064315 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Volume volume-301a11d9-13b7-459f-90cd-f479e483c213 (301a11d9-13b7-459f-90cd-f479e483c213): created successfully Oct 01 21:13:37.070154 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bd382eb4-badc-441a-b905-04591035134e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:37.079977 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Flow 'volume_create_manager' (b04e53db-5eb1-4235-9b43-194dac78bdde) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:37.102817 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-931235d8-7da1-4ed7-a7b2-3f8278c8ebd7 tempest-VolumesDeleteCascade-1038136360 None] Created volume successfully. Oct 01 21:13:37.444377 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-717980c7-8630-4b34-8982-06ca64a0f687 req-fc6baa98-5436-4305-befe-c9e407c39c44 tempest-AttachVolumeNegativeTest-855754723 None] Terminate volume connection completed successfully. Oct 01 21:13:37.444649 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-717980c7-8630-4b34-8982-06ca64a0f687 req-fc6baa98-5436-4305-befe-c9e407c39c44 tempest-AttachVolumeNegativeTest-855754723 None] Deleting attachment 95a95e19-cfbc-4f61-a817-f9547fe81247. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:13:37.800589 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6ff24bba-653b-4042-879d-4065e189116c tempest-VolumesDeleteCascade-1038136360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:38.425125 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6ff24bba-653b-4042-879d-4065e189116c tempest-VolumesDeleteCascade-1038136360 None] Create snapshot completed successfully Oct 01 21:13:39.314960 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4f683f80-8f7e-4620-829e-00f2fe2dd1b4 tempest-VolumesDeleteCascade-1038136360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:39.687811 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:39.718459 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Flow 'volume_create_manager' (845f40ee-a1e6-4fb8-819b-581fadefa87e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:39.725843 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f0f43902-0604-4070-a7bd-7b6de6728f35) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:39.742325 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f0f43902-0604-4070-a7bd-7b6de6728f35) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:39Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV355Test-Volume-926152528',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=39cc9e17-a0ec-4cdd-a684-902492ee9615,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='99af99b8903e4bcdb0f03bf35a6002c6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:40Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:40Z,use_quota=True,user_id='5b53ea073e824fc5ba6583298c8ad9c8',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:39.753813 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0b4138ca-ee44-43fc-a395-c000ee14e327) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:39.759327 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0b4138ca-ee44-43fc-a395-c000ee14e327) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:39.767814 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cfc0b35b-24b0-41f4-9ac8-ccd0b31cd451) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:39.772778 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cfc0b35b-24b0-41f4-9ac8-ccd0b31cd451) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '39cc9e17-a0ec-4cdd-a684-902492ee9615', 'volume_name': 'volume-39cc9e17-a0ec-4cdd-a684-902492ee9615', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:39.779676 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1cfccf49-209d-40d4-acc9-dbf27d20c956) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:39.784752 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1cfccf49-209d-40d4-acc9-dbf27d20c956) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:39.791037 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (49f80744-cbf9-4032-97bf-eed07874d784) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:39.792539 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Volume 39cc9e17-a0ec-4cdd-a684-902492ee9615: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-39cc9e17-a0ec-4cdd-a684-902492ee9615', 'volume_size': 1} Oct 01 21:13:39.793317 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] creating volume 'volume-39cc9e17-a0ec-4cdd-a684-902492ee9615' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:39.794317 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:39.942407 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:39.942808 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:39.946537 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (49f80744-cbf9-4032-97bf-eed07874d784) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-39cc9e17-a0ec-4cdd-a684-902492ee9615', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:39.955531 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (49f44c8b-1fd4-4c04-9449-c30611056f64) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:39.980485 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Volume volume-39cc9e17-a0ec-4cdd-a684-902492ee9615 (39cc9e17-a0ec-4cdd-a684-902492ee9615): created successfully Oct 01 21:13:39.985408 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (49f44c8b-1fd4-4c04-9449-c30611056f64) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:39.990878 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Flow 'volume_create_manager' (845f40ee-a1e6-4fb8-819b-581fadefa87e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:40.004754 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:INSERT=4,cinder:SELECT=59,cinder:UPDATE=35,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:40.015439 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e0a78716-556f-4d34-a10f-98980e052969 tempest-VolumesTransfersV355Test-824918725 None] Created volume successfully. Oct 01 21:13:40.472303 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4f683f80-8f7e-4620-829e-00f2fe2dd1b4 tempest-VolumesDeleteCascade-1038136360 None] Create snapshot completed successfully Oct 01 21:13:41.450002 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Acquiring lock "cinder-301a11d9-13b7-459f-90cd-f479e483c213-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:41.464276 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Lock "cinder-301a11d9-13b7-459f-90cd-f479e483c213-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:41.547180 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Performing cascade delete. {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/manager.py:1008}} Oct 01 21:13:41.561660 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Acquiring lock "cinder-64b284d1-082d-4d51-b02f-bfeca1d06926-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:41.577217 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Lock "cinder-64b284d1-082d-4d51-b02f-bfeca1d06926-delete_snapshot" acquired by "delete_snapshot" :: waited 0.016s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:41.593958 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:41.681235 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5a2a6f97-5b5d-438e-b2e1-344899d08082 tempest-VolumesTransfersV355Test-361094631 None] Transfer volume completed successfully. Oct 01 21:13:42.364350 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] Acquiring lock "cinder-39cc9e17-a0ec-4cdd-a684-902492ee9615-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:42.404424 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] Lock "cinder-39cc9e17-a0ec-4cdd-a684-902492ee9615-delete_volume" acquired by "delete_volume" :: waited 0.041s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:42.498112 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:42.589198 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:42.591173 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] Volume volume-39cc9e17-a0ec-4cdd-a684-902492ee9615 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:42.599397 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] deleting rbd volume volume-39cc9e17-a0ec-4cdd-a684-902492ee9615 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:42.672479 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Created reservations ['0d67b818-5747-44ac-9cdd-4789523b9ec2', 'fc0e8b18-3e43-443b-b42a-8b89197cd2ce', '7455c8b1-47c8-4400-9ece-5ce748650e41', '6b930947-15ba-4163-9fb9-34f4dd81bb41'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:42.737135 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Delete snapshot completed successfully. Oct 01 21:13:42.745050 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Lock "cinder-64b284d1-082d-4d51-b02f-bfeca1d06926-delete_snapshot" released by "delete_snapshot" :: held 1.168s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:42.746160 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Acquiring lock "cinder-d6a88b41-77b1-4410-a00c-bb02298d216a-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:42.757404 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Lock "cinder-d6a88b41-77b1-4410-a00c-bb02298d216a-delete_snapshot" acquired by "delete_snapshot" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:42.772686 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:42.901898 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] Created reservations ['6e4147c1-dc86-4f5b-92b4-5ca881f8c9a8', 'fa93f15e-8b84-4a51-b890-cca5f948d3aa', 'a930396d-1f05-4833-9b2c-c33f4c5001fd', 'ef094070-8412-46b0-acd6-6649af9eba33'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:42.969841 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:43.027616 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:43.099281 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:43.108314 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] Deleted volume successfully. Oct 01 21:13:43.134935 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-91e31528-0115-4992-8e96-6b9ef34e5a4b tempest-VolumesTransfersV355Test-878505553 None] Lock "cinder-39cc9e17-a0ec-4cdd-a684-902492ee9615-delete_volume" released by "delete_volume" :: held 0.730s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:43.510296 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Created reservations ['146d504c-b167-420e-8089-1d93aa4a4157', 'e61a00d8-9b4c-47f2-a8db-10fd4a113b69', '04b56eb2-9941-46cf-afaa-893402183a16', 'abd2cb9a-7889-432b-bc52-8b9d98da83bc'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:43.530702 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=69,cinder:UPDATE=22,cinder:DELETE=4,cinder:INSERT=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:43.585605 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Delete snapshot completed successfully. Oct 01 21:13:43.592662 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Lock "cinder-d6a88b41-77b1-4410-a00c-bb02298d216a-delete_snapshot" released by "delete_snapshot" :: held 0.835s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:43.593976 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Snapshots deleted, issuing volume delete {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/manager.py:1023}} Oct 01 21:13:43.594917 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:43.651619 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:43.652055 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Volume volume-301a11d9-13b7-459f-90cd-f479e483c213 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:43.658263 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] deleting rbd volume volume-301a11d9-13b7-459f-90cd-f479e483c213 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:44.007204 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Created reservations ['eccb1411-8329-47e5-8d43-141f8bcb4c3d', 'e003ae28-443e-48da-be9c-715ee208b5bd', '6533bb0f-f8be-4020-b7e3-c8ec1d245594', 'cb1387b0-f419-4ffd-8ec7-36cc6b5ee120'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:44.156295 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:44.176749 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Flow 'volume_create_manager' (367babf3-5dc0-4bc7-b606-f71ad2c7a5c9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:44.183318 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5ad65330-9018-46ed-82c5-2aeae21da619) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:44.195906 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:44.236528 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5ad65330-9018-46ed-82c5-2aeae21da619) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV355Test-Volume-446427229',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3efe33a4-0e0e-47d7-a14b-10810de91e53,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='99af99b8903e4bcdb0f03bf35a6002c6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:44Z,use_quota=True,user_id='5b53ea073e824fc5ba6583298c8ad9c8',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:44.243995 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2c7b6143-9872-47d5-b0db-be48e330fe15) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:44.248868 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2c7b6143-9872-47d5-b0db-be48e330fe15) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:44.253980 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8dd62ae7-808c-4fb7-a995-b0e0df8c34ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:44.258027 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8dd62ae7-808c-4fb7-a995-b0e0df8c34ca) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '3efe33a4-0e0e-47d7-a14b-10810de91e53', 'volume_name': 'volume-3efe33a4-0e0e-47d7-a14b-10810de91e53', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:44.263566 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4aa99fdd-ce66-40f0-ab09-10b4fcd5889d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:44.267298 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4aa99fdd-ce66-40f0-ab09-10b4fcd5889d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:44.271882 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8689b407-919c-4f6d-8ded-5d4673bb2503) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:44.272926 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Volume 3efe33a4-0e0e-47d7-a14b-10810de91e53: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-3efe33a4-0e0e-47d7-a14b-10810de91e53', 'volume_size': 1} Oct 01 21:13:44.273317 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] creating volume 'volume-3efe33a4-0e0e-47d7-a14b-10810de91e53' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:44.274048 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:44.307669 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:44.347650 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:44.366989 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:44.367727 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:44.371857 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8689b407-919c-4f6d-8ded-5d4673bb2503) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3efe33a4-0e0e-47d7-a14b-10810de91e53', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:44.376349 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4a4e3e2d-ac36-4d6e-9de4-ab1c9f4d200b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:44.385605 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Deleted volume successfully. Oct 01 21:13:44.401914 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Volume volume-3efe33a4-0e0e-47d7-a14b-10810de91e53 (3efe33a4-0e0e-47d7-a14b-10810de91e53): created successfully Oct 01 21:13:44.410181 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4a4e3e2d-ac36-4d6e-9de4-ab1c9f4d200b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:44.420636 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Flow 'volume_create_manager' (367babf3-5dc0-4bc7-b606-f71ad2c7a5c9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:44.434279 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1a7f735d-0472-4a9b-a2c6-c2666abf50a7 tempest-VolumesDeleteCascade-1038136360 None] Lock "cinder-301a11d9-13b7-459f-90cd-f479e483c213-delete_volume" released by "delete_volume" :: held 2.970s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:44.436365 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0c6bc0be-59cc-43b2-8e4a-43255b66182c tempest-VolumesTransfersV355Test-824918725 None] Created volume successfully. Oct 01 21:13:45.740760 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] Acquiring lock "cinder-3efe33a4-0e0e-47d7-a14b-10810de91e53-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:45.769091 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] Lock "cinder-3efe33a4-0e0e-47d7-a14b-10810de91e53-delete_volume" acquired by "delete_volume" :: waited 0.028s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:45.871871 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:45.973896 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:45.974324 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] Volume volume-3efe33a4-0e0e-47d7-a14b-10810de91e53 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:45.985321 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] deleting rbd volume volume-3efe33a4-0e0e-47d7-a14b-10810de91e53 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:46.289733 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=54,cinder:UPDATE=34,cinder:DELETE=3,cinder:INSERT=9 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:46.323096 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] Created reservations ['f7c88350-1290-4e2e-9574-a04363528ee9', '83584c48-0e06-48ae-a486-1933e802c21d', '9661d6ff-939f-4ac0-a3a1-7805dd40883b', 'd8d33629-0990-4d2a-96d8-73dd5c269f9f'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:46.396110 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:46.462217 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:46.529095 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:46.534062 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] Deleted volume successfully. Oct 01 21:13:46.554100 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4bed32f7-24cb-401d-8bbb-e527cf18e8ea tempest-VolumesTransfersV355Test-878505553 None] Lock "cinder-3efe33a4-0e0e-47d7-a14b-10810de91e53-delete_volume" released by "delete_volume" :: held 0.785s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:52.205820 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:52.250872 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Flow 'volume_create_manager' (1fdf3f82-d688-4d52-b1f4-7013408810c6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:52.269202 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5f33dd12-7978-4404-a951-5d1f298ba438) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:52.305376 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5f33dd12-7978-4404-a951-5d1f298ba438) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:52Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV357Test-Volume-450224429',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='06d9722f2159421eac66bffde6d5a901',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:52Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:52Z,use_quota=True,user_id='5eab4b929e0f4c0db0a5e48245d3df4b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:52.312584 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9309fd0e-754a-4be9-87ec-96fca7527762) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:52.317896 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9309fd0e-754a-4be9-87ec-96fca7527762) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:52.324727 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fe1b62e7-28e0-4bc2-b94c-d72d12e968f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:52.329534 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fe1b62e7-28e0-4bc2-b94c-d72d12e968f9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf', 'volume_name': 'volume-ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:52.335748 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b8cdd4c2-ce8e-4f80-8515-2a1c0f58968d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:52.340527 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b8cdd4c2-ce8e-4f80-8515-2a1c0f58968d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:52.345769 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9ac1d93c-1837-4ac0-8bdb-fc55bbad07cd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:52.348004 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Volume ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf', 'volume_size': 1} Oct 01 21:13:52.348665 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] creating volume 'volume-ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:52.349741 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:52.446984 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:52.447527 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:52.451973 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9ac1d93c-1837-4ac0-8bdb-fc55bbad07cd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:52.460363 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (103695c1-5654-4631-9c0a-7f141ce6421b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:52.476522 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Volume volume-ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf (ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf): created successfully Oct 01 21:13:52.482326 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (103695c1-5654-4631-9c0a-7f141ce6421b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:52.490577 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Flow 'volume_create_manager' (1fdf3f82-d688-4d52-b1f4-7013408810c6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:52.502710 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-2db982e0-b08b-411a-b3c7-9584e212053b tempest-VolumesTransfersV357Test-987207966 None] Created volume successfully. Oct 01 21:13:54.388852 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b86b586e-050f-4ab5-b56a-505ebb392603 tempest-VolumesTransfersV357Test-1275848098 None] Transfer volume completed successfully. Oct 01 21:13:55.193764 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] Acquiring lock "cinder-ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:55.206963 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] Lock "cinder-ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:55.259576 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:55.327311 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:55.327752 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] Volume volume-ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:55.338554 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] deleting rbd volume volume-ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:55.631819 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] Created reservations ['d8c0a73c-0be6-434c-8a8a-f441ffb67f26', '3bbf3f86-e2c6-46af-a87c-298c9f22bf80', 'f9914fea-d2d1-4db5-9684-1d232fffac77', 'e2a90ce6-7be1-49aa-977d-f9354e5076d9'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:55.731850 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:55.817105 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:55.873731 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:55.878206 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] Deleted volume successfully. Oct 01 21:13:55.896629 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ad0937d9-4856-4322-8461-b4983ce66db4 tempest-VolumesTransfersV357Test-1698186288 None] Lock "cinder-ab507d74-90d8-4e8e-a1ed-74d8f5e4c7bf-delete_volume" released by "delete_volume" :: held 0.688s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:56.736534 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:56.759261 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Flow 'volume_create_manager' (ac559c5f-05ee-4df7-8183-fe5c4b7e1162) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:56.768081 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2360f50d-a86b-4caf-bac6-ed3469ef2c45) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:56.795823 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2360f50d-a86b-4caf-bac6-ed3469ef2c45) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotListTestJSON-Volume-1388081299',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=287ae392-899d-4313-9366-2b23a15de590,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a2c9800616974afa9c35148898db5316',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:57Z,use_quota=True,user_id='bf76b289d770472cb0141e987a3eeb2c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:56.801400 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4cf76a61-b21c-4858-892f-ef2928146ca8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:56.807434 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4cf76a61-b21c-4858-892f-ef2928146ca8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:56.813024 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (db80900a-1161-4c4a-93e7-98b5940b6837) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:56.817114 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (db80900a-1161-4c4a-93e7-98b5940b6837) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '287ae392-899d-4313-9366-2b23a15de590', 'volume_name': 'volume-287ae392-899d-4313-9366-2b23a15de590', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:56.821976 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e4531b28-40fe-4512-9a26-ca485157648f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:56.825910 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e4531b28-40fe-4512-9a26-ca485157648f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:56.830828 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d4bc26b7-6bc1-4cce-a5be-e9b7ea56732a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:56.831899 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Volume 287ae392-899d-4313-9366-2b23a15de590: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-287ae392-899d-4313-9366-2b23a15de590', 'volume_size': 1} Oct 01 21:13:56.832274 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] creating volume 'volume-287ae392-899d-4313-9366-2b23a15de590' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:56.833041 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:56.874031 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:13:56.906164 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Flow 'volume_create_manager' (846d71e6-5f1d-495d-a175-53481e32620f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:56.914019 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (02ecd99c-dc82-42f8-893b-33870ccd2aeb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:56.923291 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:56.923698 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:56.927652 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d4bc26b7-6bc1-4cce-a5be-e9b7ea56732a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-287ae392-899d-4313-9366-2b23a15de590', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:56.932542 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f081a92a-55bd-4a9a-b65a-bacb333b128b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:56.950308 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (02ecd99c-dc82-42f8-893b-33870ccd2aeb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:13:57Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV357Test-Volume-815230354',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=614a1f99-4486-40d2-929e-49273cc64eda,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='06d9722f2159421eac66bffde6d5a901',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:13:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:13:57Z,use_quota=True,user_id='5eab4b929e0f4c0db0a5e48245d3df4b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:56.955483 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (31f60ddc-fc17-4e7d-9966-20daa139b447) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:56.959417 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (31f60ddc-fc17-4e7d-9966-20daa139b447) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:56.965522 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6edd0fd5-b264-4b4c-9929-5d71496b0dd3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:56.968324 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6edd0fd5-b264-4b4c-9929-5d71496b0dd3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '614a1f99-4486-40d2-929e-49273cc64eda', 'volume_name': 'volume-614a1f99-4486-40d2-929e-49273cc64eda', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:56.973953 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b9f08ae6-4e81-4b6a-8f6d-c588bc1700e0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:56.977797 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b9f08ae6-4e81-4b6a-8f6d-c588bc1700e0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:56.982253 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (01fbfd12-2c82-41e3-ba72-47b9e6f79772) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:56.983335 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Volume 614a1f99-4486-40d2-929e-49273cc64eda: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-614a1f99-4486-40d2-929e-49273cc64eda', 'volume_size': 1} Oct 01 21:13:56.983744 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] creating volume 'volume-614a1f99-4486-40d2-929e-49273cc64eda' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:13:56.984378 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:57.013003 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Volume volume-287ae392-899d-4313-9366-2b23a15de590 (287ae392-899d-4313-9366-2b23a15de590): created successfully Oct 01 21:13:57.016654 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f081a92a-55bd-4a9a-b65a-bacb333b128b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:57.022854 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Flow 'volume_create_manager' (ac559c5f-05ee-4df7-8183-fe5c4b7e1162) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:57.037503 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9dc43d82-9a52-45e6-a3ab-a42f30a506fc tempest-VolumesSnapshotListTestJSON-1877178027 None] Created volume successfully. Oct 01 21:13:57.076364 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:13:57.077054 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:13:57.082702 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (01fbfd12-2c82-41e3-ba72-47b9e6f79772) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-614a1f99-4486-40d2-929e-49273cc64eda', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:57.088469 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9fe56be5-d4ba-4143-93b4-cb7784b5abee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:13:57.100573 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Volume volume-614a1f99-4486-40d2-929e-49273cc64eda (614a1f99-4486-40d2-929e-49273cc64eda): created successfully Oct 01 21:13:57.108412 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9fe56be5-d4ba-4143-93b4-cb7784b5abee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:13:57.116990 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Flow 'volume_create_manager' (846d71e6-5f1d-495d-a175-53481e32620f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:13:57.128406 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6b6ee143-0b1d-40a9-9c6a-def402d52311 tempest-VolumesTransfersV357Test-987207966 None] Created volume successfully. Oct 01 21:13:57.968005 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-67c85ea6-2691-4649-8a58-a1c1b1a371d0 tempest-VolumesSnapshotListTestJSON-1877178027 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:58.178427 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] Acquiring lock "cinder-614a1f99-4486-40d2-929e-49273cc64eda-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:13:58.193416 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] Lock "cinder-614a1f99-4486-40d2-929e-49273cc64eda-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:13:58.233632 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:58.286463 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:13:58.287020 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] Volume volume-614a1f99-4486-40d2-929e-49273cc64eda is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:13:58.292052 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] deleting rbd volume volume-614a1f99-4486-40d2-929e-49273cc64eda {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:13:58.338842 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-67c85ea6-2691-4649-8a58-a1c1b1a371d0 tempest-VolumesSnapshotListTestJSON-1877178027 None] Create snapshot completed successfully Oct 01 21:13:58.502096 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=29,cinder:SELECT=59,cinder:INSERT=8,cinder:DELETE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:13:58.557625 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] Created reservations ['4d0ee2da-0ba0-45fd-a6ba-cd70d7347b53', 'ebe8bd76-ff36-4fb4-ae41-1bfd55bc6ed0', '5dae3277-1a24-473e-823c-6af0c07b4f2b', 'dbf29faf-ae90-4eb7-a3a3-1cd36577115f'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:13:58.626181 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:58.673333 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:13:58.747857 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:13:58.752965 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] Deleted volume successfully. Oct 01 21:13:58.766043 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ff2f66ad-c547-4a8e-afac-5fd30a2dc5bf tempest-VolumesTransfersV357Test-1698186288 None] Lock "cinder-614a1f99-4486-40d2-929e-49273cc64eda-delete_volume" released by "delete_volume" :: held 0.573s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:13:59.186697 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a30acce2-5fc9-484c-a946-aba57a74bfe6 tempest-VolumesSnapshotListTestJSON-1877178027 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:00.257098 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a30acce2-5fc9-484c-a946-aba57a74bfe6 tempest-VolumesSnapshotListTestJSON-1877178027 None] Create snapshot completed successfully Oct 01 21:14:01.598009 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7f914e74-a865-444f-916d-73951c43b3e3 tempest-VolumesSnapshotListTestJSON-1877178027 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:02.283512 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=37,cinder:SELECT=56,cinder:INSERT=4,cinder:DELETE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:02.308255 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7f914e74-a865-444f-916d-73951c43b3e3 tempest-VolumesSnapshotListTestJSON-1877178027 None] Create snapshot completed successfully Oct 01 21:14:03.432771 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f7b2310a-212e-4cd1-8c9f-494e82ee20f1 tempest-VolumesSnapshotListTestJSON-1877178027 None] Acquiring lock "cinder-a8b77abc-3b3b-469a-8da4-00d980a09ae0-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:03.447738 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f7b2310a-212e-4cd1-8c9f-494e82ee20f1 tempest-VolumesSnapshotListTestJSON-1877178027 None] Lock "cinder-a8b77abc-3b3b-469a-8da4-00d980a09ae0-delete_snapshot" acquired by "delete_snapshot" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:03.451162 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f7b2310a-212e-4cd1-8c9f-494e82ee20f1 tempest-VolumesSnapshotListTestJSON-1877178027 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:03.672699 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:14:03.692670 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Flow 'volume_create_manager' (25a5018c-91c6-485d-99ba-a991a3e6249c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:14:03.698923 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (414187a2-1d7c-42fb-b60b-99d58326af05) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:03.717146 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (414187a2-1d7c-42fb-b60b-99d58326af05) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:14:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-922284410',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=866e2465-4b6d-4443-a7bd-548f92446f52,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='817ad1c30bfc4b9c9030f36c70e07781',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:14:04Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:14:04Z,use_quota=True,user_id='b049b104d5d6487da4849ef06cda8075',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:03.722723 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (18fefc4a-ec60-4be8-a75e-12bbad42620b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:03.726972 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (18fefc4a-ec60-4be8-a75e-12bbad42620b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:03.732527 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a6fa57e6-3f1b-4d09-8eaf-7384307a7ab5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:03.736473 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a6fa57e6-3f1b-4d09-8eaf-7384307a7ab5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '866e2465-4b6d-4443-a7bd-548f92446f52', 'volume_name': 'volume-866e2465-4b6d-4443-a7bd-548f92446f52', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:03.741175 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c9ed679d-82b8-4f1a-8874-799d04e57136) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:03.745014 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c9ed679d-82b8-4f1a-8874-799d04e57136) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:03.749559 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6a5f2919-ba1d-4a57-84f0-f5fe352a4dcd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:03.750647 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Volume 866e2465-4b6d-4443-a7bd-548f92446f52: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-866e2465-4b6d-4443-a7bd-548f92446f52', 'volume_size': 1} Oct 01 21:14:03.751050 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] creating volume 'volume-866e2465-4b6d-4443-a7bd-548f92446f52' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:14:03.751854 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:03.814244 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:14:03.814681 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:14:03.818414 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6a5f2919-ba1d-4a57-84f0-f5fe352a4dcd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-866e2465-4b6d-4443-a7bd-548f92446f52', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:03.823856 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (133dd88d-d6f3-4ddf-9c25-e910f76aefc7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:03.833998 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Volume volume-866e2465-4b6d-4443-a7bd-548f92446f52 (866e2465-4b6d-4443-a7bd-548f92446f52): created successfully Oct 01 21:14:03.838430 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (133dd88d-d6f3-4ddf-9c25-e910f76aefc7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:03.847809 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Flow 'volume_create_manager' (25a5018c-91c6-485d-99ba-a991a3e6249c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:14:03.858351 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-33328c46-0ad6-47b2-b9fc-fc939e8120b3 tempest-VolumesListTestJSON-1349689719 None] Created volume successfully. Oct 01 21:14:04.365646 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-f7b2310a-212e-4cd1-8c9f-494e82ee20f1 tempest-VolumesSnapshotListTestJSON-1877178027 None] Created reservations ['8cdfb676-8aef-4ea8-9a56-f47eecca448b', '9c3f5d79-eb98-4b73-a96c-abd0a2428d77', '21cd40e5-abed-46d9-a110-603f8b7c6fb2', '517d55f9-6b19-42f3-a45e-ce62c9264a04'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:14:04.411752 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f7b2310a-212e-4cd1-8c9f-494e82ee20f1 tempest-VolumesSnapshotListTestJSON-1877178027 None] Delete snapshot completed successfully. Oct 01 21:14:04.420767 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f7b2310a-212e-4cd1-8c9f-494e82ee20f1 tempest-VolumesSnapshotListTestJSON-1877178027 None] Lock "cinder-a8b77abc-3b3b-469a-8da4-00d980a09ae0-delete_snapshot" released by "delete_snapshot" :: held 0.973s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:14:04.570863 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-61b408b0-cf42-4d68-b616-6339b7e82a4f tempest-VolumesSnapshotListTestJSON-1877178027 None] Acquiring lock "cinder-856a0c9f-a68c-47ab-9047-32ead8ccc47a-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:04.584712 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-61b408b0-cf42-4d68-b616-6339b7e82a4f tempest-VolumesSnapshotListTestJSON-1877178027 None] Lock "cinder-856a0c9f-a68c-47ab-9047-32ead8ccc47a-delete_snapshot" acquired by "delete_snapshot" :: waited 0.014s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:04.587258 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-61b408b0-cf42-4d68-b616-6339b7e82a4f tempest-VolumesSnapshotListTestJSON-1877178027 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:05.219840 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:14:05.258297 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Flow 'volume_create_manager' (6c27d00e-a08c-4757-9991-1ef9b21c54c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:14:05.266548 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (67d69c5a-a651-429e-b06e-552b42991534) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:05.288594 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (67d69c5a-a651-429e-b06e-552b42991534) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:14:05Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-598983313',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=fbee10d8-f0f3-4c44-be03-8f34bfea6f4e,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='817ad1c30bfc4b9c9030f36c70e07781',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:14:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:14:05Z,use_quota=True,user_id='b049b104d5d6487da4849ef06cda8075',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:05.299293 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (bc0454ce-ae6a-44e8-9593-79715f1ddc07) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:05.308754 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (bc0454ce-ae6a-44e8-9593-79715f1ddc07) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:05.315023 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (05853ad4-9a48-47f2-891e-812d148d6770) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:05.320715 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (05853ad4-9a48-47f2-891e-812d148d6770) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'fbee10d8-f0f3-4c44-be03-8f34bfea6f4e', 'volume_name': 'volume-fbee10d8-f0f3-4c44-be03-8f34bfea6f4e', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:05.327562 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (01ca5ec8-316e-4162-a5fc-fba96a758bbc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:05.331618 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (01ca5ec8-316e-4162-a5fc-fba96a758bbc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:05.336553 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4b071367-79cb-4dc7-b444-0f2bdd5899dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:05.340504 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Volume fbee10d8-f0f3-4c44-be03-8f34bfea6f4e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-fbee10d8-f0f3-4c44-be03-8f34bfea6f4e', 'volume_size': 1} Oct 01 21:14:05.341190 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] creating volume 'volume-fbee10d8-f0f3-4c44-be03-8f34bfea6f4e' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:14:05.342388 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:05.426245 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:14:05.426932 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:14:05.432301 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4b071367-79cb-4dc7-b444-0f2bdd5899dd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-fbee10d8-f0f3-4c44-be03-8f34bfea6f4e', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:05.447556 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e852388a-f909-43e4-9b4f-f108b8cf465f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:05.465963 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Volume volume-fbee10d8-f0f3-4c44-be03-8f34bfea6f4e (fbee10d8-f0f3-4c44-be03-8f34bfea6f4e): created successfully Oct 01 21:14:05.470703 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e852388a-f909-43e4-9b4f-f108b8cf465f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:05.476754 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Flow 'volume_create_manager' (6c27d00e-a08c-4757-9991-1ef9b21c54c5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:14:05.499961 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e39fad91-23dc-4606-a801-5c6dbbdca680 tempest-VolumesListTestJSON-1349689719 None] Created volume successfully. Oct 01 21:14:05.517224 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-61b408b0-cf42-4d68-b616-6339b7e82a4f tempest-VolumesSnapshotListTestJSON-1877178027 None] Created reservations ['346c8e59-67bf-447b-af6c-ef4b463e381d', 'd7b0cd76-b5a1-4654-8258-deb0050d4154', '282faf54-012e-40d7-aa17-ae99ef45b02d', '1d29e4f2-5c5d-4f6d-b6e3-cb69e3faaa2f'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:14:05.572484 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-61b408b0-cf42-4d68-b616-6339b7e82a4f tempest-VolumesSnapshotListTestJSON-1877178027 None] Delete snapshot completed successfully. Oct 01 21:14:05.578855 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-61b408b0-cf42-4d68-b616-6339b7e82a4f tempest-VolumesSnapshotListTestJSON-1877178027 None] Lock "cinder-856a0c9f-a68c-47ab-9047-32ead8ccc47a-delete_snapshot" released by "delete_snapshot" :: held 0.994s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:14:05.699597 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6852495f-1a1d-41f8-9867-0eea0c8dc58d tempest-VolumesSnapshotListTestJSON-1877178027 None] Acquiring lock "cinder-6c08dab8-2826-4f60-9ae5-b71817080099-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:05.716319 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6852495f-1a1d-41f8-9867-0eea0c8dc58d tempest-VolumesSnapshotListTestJSON-1877178027 None] Lock "cinder-6c08dab8-2826-4f60-9ae5-b71817080099-delete_snapshot" acquired by "delete_snapshot" :: waited 0.017s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:05.718954 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6852495f-1a1d-41f8-9867-0eea0c8dc58d tempest-VolumesSnapshotListTestJSON-1877178027 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:06.408233 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-6852495f-1a1d-41f8-9867-0eea0c8dc58d tempest-VolumesSnapshotListTestJSON-1877178027 None] Created reservations ['a3e91a90-b84c-4ee8-a26f-15aaf1eaddcd', 'd3b46a9f-c7a4-47cb-8921-678d7537d91a', '0c1ef78e-5d44-4bf3-a30a-0c9b3b3a318e', '3e7850ed-218b-46b5-86a2-844efd3d539f'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:14:06.452026 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6852495f-1a1d-41f8-9867-0eea0c8dc58d tempest-VolumesSnapshotListTestJSON-1877178027 None] Delete snapshot completed successfully. Oct 01 21:14:06.459325 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6852495f-1a1d-41f8-9867-0eea0c8dc58d tempest-VolumesSnapshotListTestJSON-1877178027 None] Lock "cinder-6c08dab8-2826-4f60-9ae5-b71817080099-delete_snapshot" released by "delete_snapshot" :: held 0.742s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:14:06.689446 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:14:06.708805 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Flow 'volume_create_manager' (0cca3a94-0d3c-411e-9e4d-d3510a20e420) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:14:06.714866 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6acaf3af-ca03-4138-81af-a957cf3bcc27) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:06.719207 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=57,cinder:INSERT=8,cinder:UPDATE=30,cinder:DELETE=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:06.730594 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6acaf3af-ca03-4138-81af-a957cf3bcc27) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:14:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-1692683732',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=154e96ec-41d1-4a78-9613-360c1566727a,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='817ad1c30bfc4b9c9030f36c70e07781',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:14:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:14:07Z,use_quota=True,user_id='b049b104d5d6487da4849ef06cda8075',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:06.739272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (49defc65-c670-44a6-94e9-1cef996e19d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:06.743836 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (49defc65-c670-44a6-94e9-1cef996e19d7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:06.750430 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fdbfbe40-1f90-492b-b359-7c05324ae0cf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:06.755515 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fdbfbe40-1f90-492b-b359-7c05324ae0cf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '154e96ec-41d1-4a78-9613-360c1566727a', 'volume_name': 'volume-154e96ec-41d1-4a78-9613-360c1566727a', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:06.761073 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (758098a9-67e4-4da9-a1c5-1416628663c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:06.765332 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (758098a9-67e4-4da9-a1c5-1416628663c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:06.770233 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aed75223-a96b-4114-9a2b-366be9fe7e05) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:06.771518 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Volume 154e96ec-41d1-4a78-9613-360c1566727a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-154e96ec-41d1-4a78-9613-360c1566727a', 'volume_size': 1} Oct 01 21:14:06.771986 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] creating volume 'volume-154e96ec-41d1-4a78-9613-360c1566727a' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:14:06.773239 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:06.836224 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] Acquiring lock "cinder-287ae392-899d-4313-9366-2b23a15de590-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:06.849384 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:14:06.849824 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:14:06.854082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aed75223-a96b-4114-9a2b-366be9fe7e05) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-154e96ec-41d1-4a78-9613-360c1566727a', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:06.862980 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (458ea50f-cb98-459d-b363-1b8a69b774a6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:06.873784 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] Lock "cinder-287ae392-899d-4313-9366-2b23a15de590-delete_volume" acquired by "delete_volume" :: waited 0.038s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:06.888241 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Volume volume-154e96ec-41d1-4a78-9613-360c1566727a (154e96ec-41d1-4a78-9613-360c1566727a): created successfully Oct 01 21:14:06.891943 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (458ea50f-cb98-459d-b363-1b8a69b774a6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:06.897481 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Flow 'volume_create_manager' (0cca3a94-0d3c-411e-9e4d-d3510a20e420) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:14:06.916875 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6ed14d28-5bca-47a6-8ebc-77b85e33c518 tempest-VolumesListTestJSON-1349689719 None] Created volume successfully. Oct 01 21:14:06.928791 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:06.978355 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:14:06.978845 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] Volume volume-287ae392-899d-4313-9366-2b23a15de590 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:14:06.985173 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] deleting rbd volume volume-287ae392-899d-4313-9366-2b23a15de590 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:14:07.193673 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] Created reservations ['28890cd4-b015-428e-9301-6199510f9b2c', 'c1a129b2-105b-4dec-ae08-1b656bfd2191', '49d862c4-3446-4ad8-8d8a-3a069402f51d', '9da598af-2ca1-4b5c-9c96-a7325d77ef5b'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:14:07.242082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:07.270586 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:07.299439 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:14:07.305256 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] Deleted volume successfully. Oct 01 21:14:07.318335 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-8b6d9a20-441e-4f29-83af-64e590961d14 tempest-VolumesSnapshotListTestJSON-1877178027 None] Lock "cinder-287ae392-899d-4313-9366-2b23a15de590-delete_volume" released by "delete_volume" :: held 0.445s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:14:10.406626 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] Acquiring lock "cinder-154e96ec-41d1-4a78-9613-360c1566727a-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:10.418071 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] Lock "cinder-154e96ec-41d1-4a78-9613-360c1566727a-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:10.500323 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:10.553330 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:14:10.553758 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] Volume volume-154e96ec-41d1-4a78-9613-360c1566727a is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:14:10.559834 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] deleting rbd volume volume-154e96ec-41d1-4a78-9613-360c1566727a {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:14:10.772322 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] Created reservations ['0707e36d-6d60-4cc9-af5e-ceafddc118ac', '15edda98-3c01-4610-8015-1c4a8e53f963', 'e07fcac0-ddfd-44ab-b2ad-bbb96f1550f2', '79b38783-80a2-47c4-8975-810cc4130fb9'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:14:10.799742 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=57,cinder:UPDATE=31,cinder:INSERT=12 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:10.828157 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:10.870203 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:10.915467 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:14:10.922340 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] Deleted volume successfully. Oct 01 21:14:10.937743 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a59084d8-c772-4c3b-a008-065b480dabf9 tempest-VolumesListTestJSON-1349689719 None] Lock "cinder-154e96ec-41d1-4a78-9613-360c1566727a-delete_volume" released by "delete_volume" :: held 0.520s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:14:11.564802 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] Acquiring lock "cinder-fbee10d8-f0f3-4c44-be03-8f34bfea6f4e-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:11.573975 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] Lock "cinder-fbee10d8-f0f3-4c44-be03-8f34bfea6f4e-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:11.615542 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:11.689487 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:14:11.690004 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] Volume volume-fbee10d8-f0f3-4c44-be03-8f34bfea6f4e is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:14:11.695217 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] deleting rbd volume volume-fbee10d8-f0f3-4c44-be03-8f34bfea6f4e {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:14:11.932686 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] Created reservations ['a5418a1f-e449-4bff-b2b2-28760949c86f', '7b915188-eebb-471f-902b-316a289a7f37', '77126752-6355-43ca-9215-894ccba582c6', '95520baa-c2b4-4d51-ba19-a56678cf1952'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:14:11.996823 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:12.040314 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:12.081931 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:14:12.088193 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] Deleted volume successfully. Oct 01 21:14:12.102598 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-50bd028a-607b-467b-9d10-c16c192df984 tempest-VolumesListTestJSON-1349689719 None] Lock "cinder-fbee10d8-f0f3-4c44-be03-8f34bfea6f4e-delete_volume" released by "delete_volume" :: held 0.528s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:14:12.755891 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] Acquiring lock "cinder-866e2465-4b6d-4443-a7bd-548f92446f52-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:12.768706 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] Lock "cinder-866e2465-4b6d-4443-a7bd-548f92446f52-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:12.812498 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:12.871087 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:14:12.871521 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] Volume volume-866e2465-4b6d-4443-a7bd-548f92446f52 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:14:12.882802 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] deleting rbd volume volume-866e2465-4b6d-4443-a7bd-548f92446f52 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:14:13.130534 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] Created reservations ['871e1e50-ca73-4665-9118-20f25ce0161b', '119bf582-0efd-424b-bf14-eab7e701983a', '772e1a07-33b9-4929-8453-11bbd2718c26', '36069aac-7c9d-4eb1-9ca2-23e819e494b0'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:14:13.180698 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=55,cinder:UPDATE=38,cinder:DELETE=2,cinder:INSERT=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:13.187902 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:13.233600 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:13.265633 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:14:13.269730 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] Deleted volume successfully. Oct 01 21:14:13.285122 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0e5b87c0-831f-4aab-b1d6-7c11772ef773 tempest-VolumesListTestJSON-1349689719 None] Lock "cinder-866e2465-4b6d-4443-a7bd-548f92446f52-delete_volume" released by "delete_volume" :: held 0.516s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:14:23.810130 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:14:23.811343 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:23.855440 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:23.895321 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:14:24.332970 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:29.333514 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] Acquiring lock "cinder-e961484c-2fba-4c14-8029-87f4dd61959d-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:29.363658 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] Lock "cinder-e961484c-2fba-4c14-8029-87f4dd61959d-delete_volume" acquired by "delete_volume" :: waited 0.030s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:29.422853 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:29.498469 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:14:29.499069 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] Volume volume-e961484c-2fba-4c14-8029-87f4dd61959d is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:14:29.509435 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] deleting rbd volume volume-e961484c-2fba-4c14-8029-87f4dd61959d {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:14:29.831166 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] Created reservations ['b58d11a5-ea1d-4939-b9bf-0c1a4344b588', '3413d4e8-d30b-4d7d-839f-ba839a132526', '2a813267-9a4c-417c-9f66-a60a2be67d41', '88283006-f3af-4ed8-9f06-0914bb685327'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:14:29.899092 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:29.963647 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:30.020507 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:14:30.025229 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] Deleted volume successfully. Oct 01 21:14:30.055345 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3c7c7da4-4475-44d0-b9ff-2e59ae03b252 tempest-AttachVolumeShelveTestJSON-705121323 None] Lock "cinder-e961484c-2fba-4c14-8029-87f4dd61959d-delete_volume" released by "delete_volume" :: held 0.692s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:14:30.651721 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] Acquiring lock "cinder-45c8ab81-bd0c-4e31-bf20-a283c57ecdab-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:30.668581 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] Lock "cinder-45c8ab81-bd0c-4e31-bf20-a283c57ecdab-delete_volume" acquired by "delete_volume" :: waited 0.017s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:30.727305 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:30.787281 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:14:30.787852 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] Volume volume-45c8ab81-bd0c-4e31-bf20-a283c57ecdab is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:14:30.794249 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] deleting rbd volume volume-45c8ab81-bd0c-4e31-bf20-a283c57ecdab {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:14:31.133094 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] Created reservations ['1fb4d01d-8013-4ba6-83bf-6e29d03a0123', '9d34112c-44fd-437d-9557-c9b649784235', '719bf355-962c-4182-b136-6d2c64dc391a', '82b13003-b844-4c2b-8871-de035a1ea8a5'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:14:31.211237 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:31.255428 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:31.327874 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:14:31.333247 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] Deleted volume successfully. Oct 01 21:14:31.356106 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-11698ade-ccd0-47ac-b021-adc1028041d7 tempest-AttachVolumeShelveTestJSON-705121323 None] Lock "cinder-45c8ab81-bd0c-4e31-bf20-a283c57ecdab-delete_volume" released by "delete_volume" :: held 0.688s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:14:41.338236 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=28,cinder:UPDATE=17,cinder:INSERT=5,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:43.616597 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:DELETE=4,cinder:SELECT=51,cinder:UPDATE=33,cinder:INSERT=8 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:14:43.633615 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:14:43.653754 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Flow 'volume_create_manager' (38682df6-71e6-4baa-9387-150ead02efec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:14:43.660948 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2931a6ce-9eb0-4bcb-bf05-2036bbf06836) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:43.684895 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2931a6ce-9eb0-4bcb-bf05-2036bbf06836) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:14:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-1372259356',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5cc6c2ae-e749-490d-86dd-a079f9551aa1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='24a06f5da36b4488a1b8ea01729b0e6c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:14:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:14:44Z,use_quota=True,user_id='902ba44d318d4a01a510f732d27467c9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:43.690024 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d0f3f472-cf9d-4b7b-afd2-8f573ebc2340) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:43.694574 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d0f3f472-cf9d-4b7b-afd2-8f573ebc2340) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:43.699580 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c578da7-4cf2-4267-a7f3-c8d9fe3484f4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:43.703711 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c578da7-4cf2-4267-a7f3-c8d9fe3484f4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5cc6c2ae-e749-490d-86dd-a079f9551aa1', 'volume_name': 'volume-5cc6c2ae-e749-490d-86dd-a079f9551aa1', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:43.708363 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (585ee4ae-f02f-42d1-98a5-a8a5a6472b4f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:43.713682 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (585ee4ae-f02f-42d1-98a5-a8a5a6472b4f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:43.718638 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (98e07f8b-9fa9-4ca2-a676-86653b382a32) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:43.719923 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Volume 5cc6c2ae-e749-490d-86dd-a079f9551aa1: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5cc6c2ae-e749-490d-86dd-a079f9551aa1', 'volume_size': 1} Oct 01 21:14:43.720891 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] creating volume 'volume-5cc6c2ae-e749-490d-86dd-a079f9551aa1' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:14:43.721818 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:43.827377 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:14:43.827931 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:14:43.833118 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (98e07f8b-9fa9-4ca2-a676-86653b382a32) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5cc6c2ae-e749-490d-86dd-a079f9551aa1', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:43.838477 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (81e226ea-a1c2-4682-9d53-d6e62ee24715) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:14:43.849927 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Volume volume-5cc6c2ae-e749-490d-86dd-a079f9551aa1 (5cc6c2ae-e749-490d-86dd-a079f9551aa1): created successfully Oct 01 21:14:43.853740 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (81e226ea-a1c2-4682-9d53-d6e62ee24715) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:14:43.859634 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Flow 'volume_create_manager' (38682df6-71e6-4baa-9387-150ead02efec) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:14:43.871216 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-cb5d6989-c26e-44f9-9f47-c5df5bcd4c27 tempest-AttachVolumeNegativeTest-855754723 None] Created volume successfully. Oct 01 21:14:45.273748 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-49305d10-9446-452f-816e-8797213f7c0b req-31f5b563-fbec-4369-9b04-d68237e1162a tempest-AttachVolumeNegativeTest-855754723 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:14:46.151780 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-49305d10-9446-452f-816e-8797213f7c0b req-31f5b563-fbec-4369-9b04-d68237e1162a tempest-AttachVolumeNegativeTest-855754723 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.875s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:14:46.154220 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-49305d10-9446-452f-816e-8797213f7c0b req-31f5b563-fbec-4369-9b04-d68237e1162a tempest-AttachVolumeNegativeTest-855754723 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-5cc6c2ae-e749-490d-86dd-a079f9551aa1', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '5cc6c2ae-e749-490d-86dd-a079f9551aa1', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:14:46.231746 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-49305d10-9446-452f-816e-8797213f7c0b req-31f5b563-fbec-4369-9b04-d68237e1162a tempest-AttachVolumeNegativeTest-855754723 None] Connection info returned from driver {'name': 'volumes/volume-5cc6c2ae-e749-490d-86dd-a079f9551aa1', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '5cc6c2ae-e749-490d-86dd-a079f9551aa1', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '36ed2045-6593-4e6e-b9be-e7b487a9f18b'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:14:46.309444 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-49305d10-9446-452f-816e-8797213f7c0b req-31f5b563-fbec-4369-9b04-d68237e1162a tempest-AttachVolumeNegativeTest-855754723 None] attachment_update completed successfully. Oct 01 21:14:49.700351 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-824331d8-468a-442c-8a77-b25d4df9907f req-c8742dd4-ae31-427e-9c88-5014136dc8f2 tempest-AttachVolumeNegativeTest-855754723 None] Terminate volume connection completed successfully. Oct 01 21:14:49.700894 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-824331d8-468a-442c-8a77-b25d4df9907f req-c8742dd4-ae31-427e-9c88-5014136dc8f2 tempest-AttachVolumeNegativeTest-855754723 None] Deleting attachment 36ed2045-6593-4e6e-b9be-e7b487a9f18b. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:14:57.222085 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] Acquiring lock "cinder-5cc6c2ae-e749-490d-86dd-a079f9551aa1-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:57.264006 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] Lock "cinder-5cc6c2ae-e749-490d-86dd-a079f9551aa1-delete_volume" acquired by "delete_volume" :: waited 0.041s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:57.339503 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:57.431255 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:14:57.431766 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] Volume volume-5cc6c2ae-e749-490d-86dd-a079f9551aa1 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:14:57.441437 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] deleting rbd volume volume-5cc6c2ae-e749-490d-86dd-a079f9551aa1 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:14:57.672305 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] Created reservations ['d6e50d4d-f2c2-4c40-82e3-f4c22498e320', '4ffc9e3e-96b8-4688-af36-af9b8c3340c0', '924d09af-6e6f-473f-95ca-c966e75ab7d4', '32a1211a-d65a-4532-a38b-41f79ddf8f24'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:14:57.720533 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:57.762960 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:57.790971 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:14:57.804753 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] Deleted volume successfully. Oct 01 21:14:57.808666 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c7667afd-0408-449f-a6f7-479f0d83fed1 tempest-AttachVolumeNegativeTest-855754723 None] Lock "cinder-5cc6c2ae-e749-490d-86dd-a079f9551aa1-delete_volume" released by "delete_volume" :: held 0.545s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:14:58.486262 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] Acquiring lock "cinder-46afd2fc-5506-4c1c-8c97-203abd3a41d9-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:58.513272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] Lock "cinder-46afd2fc-5506-4c1c-8c97-203abd3a41d9-delete_volume" acquired by "delete_volume" :: waited 0.027s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:58.563057 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:58.637965 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:14:58.638716 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] Volume volume-46afd2fc-5506-4c1c-8c97-203abd3a41d9 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:14:58.643223 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] deleting rbd volume volume-46afd2fc-5506-4c1c-8c97-203abd3a41d9 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:14:58.877136 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] Created reservations ['cce4d4ce-a854-432f-8a8a-006f6fa966ac', '39f62d67-e679-4737-8c59-9bb4414f61aa', 'c4e98f15-79b7-4a0d-9428-aa7bc3811317', 'f405fbd6-c13b-4a30-a4f7-f04107232b8b'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:14:58.948137 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:58.985182 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:14:59.051650 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:14:59.059852 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] Deleted volume successfully. Oct 01 21:14:59.074647 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e978f553-1774-4b3e-9f47-e4cb0eb07121 tempest-AttachVolumeNegativeTest-855754723 None] Lock "cinder-46afd2fc-5506-4c1c-8c97-203abd3a41d9-delete_volume" released by "delete_volume" :: held 0.561s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:14:59.890759 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] Acquiring lock "cinder-07294dd9-ac7c-41ee-a684-07fa45f5d09a-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:14:59.902244 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] Lock "cinder-07294dd9-ac7c-41ee-a684-07fa45f5d09a-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:14:59.993318 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:00.075931 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:15:00.076917 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] Volume volume-07294dd9-ac7c-41ee-a684-07fa45f5d09a is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:15:00.085166 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] deleting rbd volume volume-07294dd9-ac7c-41ee-a684-07fa45f5d09a {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:15:00.321507 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] Created reservations ['2f1710cb-d76c-460c-817d-30c91df586c2', '13d4e31e-c998-42bc-9a1d-cc4e15ce5476', '96fbc31a-9a0d-4c33-817b-16ff6b02c80a', 'a2c28047-1cf5-4e85-8397-1a485dcc5535'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:15:00.388507 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:00.444389 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:00.482933 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:15:00.492711 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] Deleted volume successfully. Oct 01 21:15:00.508430 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0b41da36-5c30-4067-8f1c-54e8781492f6 tempest-AttachVolumeNegativeTest-855754723 None] Lock "cinder-07294dd9-ac7c-41ee-a684-07fa45f5d09a-delete_volume" released by "delete_volume" :: held 0.606s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:15:04.753837 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:15:04.780237 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Flow 'volume_create_manager' (ca3aeee2-af5a-4c72-b875-5d250bc93755) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:15:04.788356 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6980b55e-1f04-481e-a541-06d28681ee2e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:04.823072 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6980b55e-1f04-481e-a541-06d28681ee2e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:15:05Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesExtendTest-Volume-624277817',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=428a311d-5c53-459f-976f-19642ba9b9e3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='aa31359109754fcba8ad848702356b65',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:15:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:15:05Z,use_quota=True,user_id='39af6e701f564a2888b09d1eafe3478d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:04.830477 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5a999317-5408-4b03-8b9e-d2f2c3bac7b2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:04.835185 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5a999317-5408-4b03-8b9e-d2f2c3bac7b2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:04.845059 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e8702eac-d4d3-43d7-9dcc-15e6dac48ecf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:04.948579 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e8702eac-d4d3-43d7-9dcc-15e6dac48ecf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '428a311d-5c53-459f-976f-19642ba9b9e3', 'volume_name': 'volume-428a311d-5c53-459f-976f-19642ba9b9e3', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:04.952991 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ba71118b-a642-43f6-bde2-3322487f42bc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:04.956977 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ba71118b-a642-43f6-bde2-3322487f42bc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:04.961917 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2359073c-20a5-42c6-83db-3fbd6d0fd80d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:04.963281 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Volume 428a311d-5c53-459f-976f-19642ba9b9e3: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-428a311d-5c53-459f-976f-19642ba9b9e3', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:15:04.964199 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Cloning 428a311d-5c53-459f-976f-19642ba9b9e3 from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:15:04.965498 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:15:04.974979 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:15:04.980861 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=67,cinder:UPDATE=22,cinder:DELETE=2,cinder:INSERT=9 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:04.987827 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 15, 4, 979698)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:15:04.988270 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:15:04.988717 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:15:04.995421 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.021s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:15:04.996586 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:15:05.003203 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 15, 5, 372)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:15:05.003615 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:15:05.004195 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:15:05.007410 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=57,cinder:UPDATE=38,cinder:DELETE=2,cinder:INSERT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:05.018657 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:05.086477 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] creating snapshot='volume-428a311d-5c53-459f-976f-19642ba9b9e3.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:15:05.315202 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-428a311d-5c53-459f-976f-19642ba9b9e3.clone_snap' to 'volume-428a311d-5c53-459f-976f-19642ba9b9e3' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:15:05.471556 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:15:05.473298 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:15:05.473670 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:15:05.495757 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:15:05.496739 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Marking volume 428a311d-5c53-459f-976f-19642ba9b9e3 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:15:05.508229 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to 428a311d-5c53-459f-976f-19642ba9b9e3. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:15:05.509325 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Creating volume glance metadata for volume 428a311d-5c53-459f-976f-19642ba9b9e3 backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:15:05.638860 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2359073c-20a5-42c6-83db-3fbd6d0fd80d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-428a311d-5c53-459f-976f-19642ba9b9e3', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:05.644441 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9c45091d-c353-45fd-aa84-ba249325f91e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:05.655825 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Volume volume-428a311d-5c53-459f-976f-19642ba9b9e3 (428a311d-5c53-459f-976f-19642ba9b9e3): created successfully Oct 01 21:15:05.663891 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9c45091d-c353-45fd-aa84-ba249325f91e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:05.680398 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Flow 'volume_create_manager' (ca3aeee2-af5a-4c72-b875-5d250bc93755) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:15:05.696289 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0cc8496a-3698-4ca9-bd75-79b0274b82f1 tempest-VolumesExtendTest-1576879390 None] Created volume successfully. Oct 01 21:15:05.963669 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-63cb6cde-bba8-42ec-9fb7-f45b6e9c7000 tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:06.132766 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-63cb6cde-bba8-42ec-9fb7-f45b6e9c7000 tempest-VolumesExtendTest-1576879390 None] Extend volume from 1 GB to 2 GB. {{(pid=133391) extend_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:2012}} Oct 01 21:15:06.163570 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-63cb6cde-bba8-42ec-9fb7-f45b6e9c7000 tempest-VolumesExtendTest-1576879390 None] Extend volume completed successfully. Oct 01 21:15:07.545081 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:15:07.564500 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Flow 'volume_create_manager' (18098b36-85fb-4eae-a89a-da6c7ec5f5d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:15:07.571196 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (349ac620-66ec-4466-a76a-23ee32ac7a12) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:07.587539 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (349ac620-66ec-4466-a76a-23ee32ac7a12) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:15:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesExtendTest-Volume-682325990',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ccfaf696-cc17-42a9-90ed-fba107da2a75,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='aa31359109754fcba8ad848702356b65',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:15:08Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:15:08Z,use_quota=True,user_id='39af6e701f564a2888b09d1eafe3478d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:07.592637 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (24f72470-33d5-4f50-9554-5da4ad9f387c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:07.596574 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (24f72470-33d5-4f50-9554-5da4ad9f387c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:07.601962 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (94c142b6-cfca-46f2-8dcf-631d3d429c69) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:07.605832 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (94c142b6-cfca-46f2-8dcf-631d3d429c69) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ccfaf696-cc17-42a9-90ed-fba107da2a75', 'volume_name': 'volume-ccfaf696-cc17-42a9-90ed-fba107da2a75', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:07.611009 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b5415279-9076-47f4-89b9-8ec5aa6631de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:07.615125 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b5415279-9076-47f4-89b9-8ec5aa6631de) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:07.622695 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4da2ffb3-0818-4af0-9bbc-23da7d927c38) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:07.625716 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Volume ccfaf696-cc17-42a9-90ed-fba107da2a75: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ccfaf696-cc17-42a9-90ed-fba107da2a75', 'volume_size': 1} Oct 01 21:15:07.626158 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] creating volume 'volume-ccfaf696-cc17-42a9-90ed-fba107da2a75' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:15:07.626989 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:07.687659 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:15:07.689843 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:15:07.693677 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4da2ffb3-0818-4af0-9bbc-23da7d927c38) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ccfaf696-cc17-42a9-90ed-fba107da2a75', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:07.698032 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (84d5e2d9-a604-4b54-87cd-f59630bf3eff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:15:07.711330 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Volume volume-ccfaf696-cc17-42a9-90ed-fba107da2a75 (ccfaf696-cc17-42a9-90ed-fba107da2a75): created successfully Oct 01 21:15:07.715867 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (84d5e2d9-a604-4b54-87cd-f59630bf3eff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:15:07.722179 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Flow 'volume_create_manager' (18098b36-85fb-4eae-a89a-da6c7ec5f5d0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:15:07.736531 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f7c9c0d2-f243-4464-962e-981c1763d30f tempest-VolumesExtendTest-1576879390 None] Created volume successfully. Oct 01 21:15:08.839144 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6acae1f4-3802-47ac-8d37-590d390aef1d tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:09.373147 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6acae1f4-3802-47ac-8d37-590d390aef1d tempest-VolumesExtendTest-1576879390 None] Create snapshot completed successfully Oct 01 21:15:10.005062 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-092d9786-06ef-40d0-ba20-bce3787265da tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:10.107486 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-092d9786-06ef-40d0-ba20-bce3787265da tempest-VolumesExtendTest-1576879390 None] Extend volume from 1 GB to 2 GB. {{(pid=133391) extend_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:2012}} Oct 01 21:15:10.138774 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-092d9786-06ef-40d0-ba20-bce3787265da tempest-VolumesExtendTest-1576879390 None] Extend volume completed successfully. Oct 01 21:15:11.250596 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-28fd5ed6-1d94-4f79-a5dd-c298d30295b0 tempest-VolumesExtendTest-1576879390 None] Acquiring lock "cinder-21a5813f-2e6f-4548-87c6-74428bcbf3b9-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:15:11.264517 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-28fd5ed6-1d94-4f79-a5dd-c298d30295b0 tempest-VolumesExtendTest-1576879390 None] Lock "cinder-21a5813f-2e6f-4548-87c6-74428bcbf3b9-delete_snapshot" acquired by "delete_snapshot" :: waited 0.014s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:15:11.266706 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-28fd5ed6-1d94-4f79-a5dd-c298d30295b0 tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:12.472849 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-28fd5ed6-1d94-4f79-a5dd-c298d30295b0 tempest-VolumesExtendTest-1576879390 None] Created reservations ['19fa1c22-8399-4de0-bbfd-9919424e5e76', 'e01f1e96-1f38-4f4b-b72e-0e045d0d9a24', '731bb561-b1b6-499d-bae6-77f9bd3ba137', '6f3526f2-05cc-4fb4-84b6-8ca34fd6af4f'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:15:12.529663 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-28fd5ed6-1d94-4f79-a5dd-c298d30295b0 tempest-VolumesExtendTest-1576879390 None] Delete snapshot completed successfully. Oct 01 21:15:12.537908 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-28fd5ed6-1d94-4f79-a5dd-c298d30295b0 tempest-VolumesExtendTest-1576879390 None] Lock "cinder-21a5813f-2e6f-4548-87c6-74428bcbf3b9-delete_snapshot" released by "delete_snapshot" :: held 1.273s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:15:13.435032 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] Acquiring lock "cinder-ccfaf696-cc17-42a9-90ed-fba107da2a75-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:15:13.450008 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] Lock "cinder-ccfaf696-cc17-42a9-90ed-fba107da2a75-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:15:13.523106 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:13.616502 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:15:13.617051 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] Volume volume-ccfaf696-cc17-42a9-90ed-fba107da2a75 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:15:13.635421 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] deleting rbd volume volume-ccfaf696-cc17-42a9-90ed-fba107da2a75 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:15:13.960751 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] Created reservations ['335e8f76-cd27-483f-8348-3a7ac1eb74ee', '2f9c0a7d-d5bf-4681-8b5e-0be4ff92842f', '78608267-546c-4078-82eb-ed828da7d182', 'f557f50a-d0f9-4fc7-8fb9-18f9bc6667c7'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:15:14.113408 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:14.190822 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:14.256974 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:15:14.263679 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] Deleted volume successfully. Oct 01 21:15:14.279547 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ab7d3592-f79c-4f76-a0f2-616d59fceb1c tempest-VolumesExtendTest-1576879390 None] Lock "cinder-ccfaf696-cc17-42a9-90ed-fba107da2a75-delete_volume" released by "delete_volume" :: held 0.830s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:15:14.641226 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] Acquiring lock "cinder-428a311d-5c53-459f-976f-19642ba9b9e3-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:15:14.656518 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] Lock "cinder-428a311d-5c53-459f-976f-19642ba9b9e3-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:15:14.783534 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:14.908949 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:15:14.924248 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] deleting rbd volume volume-428a311d-5c53-459f-976f-19642ba9b9e3 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:15:15.288137 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:15:15.330944 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] deleting parent snapshot volume-428a311d-5c53-459f-976f-19642ba9b9e3.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:15:16.496079 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] Created reservations ['51b60b3c-53f6-431e-8b5d-575eab9ae038', '674435ab-4a9f-4fd3-8113-8e46ed12adde', '7d8c79fd-a80b-436b-8db7-0fefd8f6023a', '0a39719f-5cd3-41aa-bba6-44bc843b008a'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:15:16.543136 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=51,cinder:INSERT=14,cinder:UPDATE=31,cinder:DELETE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:16.556927 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=27,cinder:SELECT=63,cinder:INSERT=10 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:16.590614 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:16.652265 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:16.714136 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:15:16.721444 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] Deleted volume successfully. Oct 01 21:15:16.737216 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5a786c67-13ad-40df-bd47-8bbfc205a809 tempest-VolumesExtendTest-1576879390 None] Lock "cinder-428a311d-5c53-459f-976f-19642ba9b9e3-delete_volume" released by "delete_volume" :: held 2.081s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:15:23.810673 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:15:23.812002 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:23.867626 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:15:23.925737 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:15:34.341062 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:15:54.344961 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=11,cinder:DELETE=1,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:00.127513 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] Acquiring lock "cinder-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:16:00.158815 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] Lock "cinder-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360-delete_volume" acquired by "delete_volume" :: waited 0.033s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:16:00.264019 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:16:00.327724 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:16:00.328427 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] Volume volume-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:16:00.336862 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] deleting rbd volume volume-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:16:00.750382 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] Created reservations ['a24aaf32-133f-4267-9917-59d079eda25b', '51ff0f0c-86c9-4f44-b0c1-0f6b04fd8073', 'aefc1da1-3176-4a0a-987e-5cb762957c63', '466af8fd-8674-4992-bc0a-56c22db35bab'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:16:00.839338 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:16:00.920462 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:16:00.986872 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:16:01.000558 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] Deleted volume successfully. Oct 01 21:16:01.022951 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4780846e-243f-452a-830b-5b858c5b3c11 tempest-DeleteServersTestJSON-611460547 None] Lock "cinder-4e46ec4a-c69b-4a8d-b661-2d1bc20ed360-delete_volume" released by "delete_volume" :: held 0.864s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:16:10.834336 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=13,cinder:UPDATE=8,cinder:INSERT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:14.349694 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=16,cinder:UPDATE=10,cinder:INSERT=1,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:23.816741 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:16:23.816741 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:16:23.898977 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:16:23.967799 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:16:34.373486 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:16:44.350892 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=9,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:04.357507 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:23.815712 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:17:23.816361 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:17:23.876626 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:17:23.948055 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:17:24.355773 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:17:44.351332 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:04.353337 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:14.367313 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=2,cinder:SELECT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:23.819014 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:18:23.820422 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:23.891649 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:23.952123 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:18:34.356305 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:34.763358 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:18:34.797267 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Flow 'volume_create_manager' (21078687-0435-4786-9f0e-6d7e2a1b4354) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:18:34.803485 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e272bf6f-e2b6-467c-83f2-3a354c18daf2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:34.819889 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e272bf6f-e2b6-467c-83f2-3a354c18daf2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:18:34Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasNegativeTestJSON-Volume-1790374871',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d7631a2d-ca72-4741-b286-e081bbc5db7a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2cc0a640e91c4ca8ac4593d20fc4114c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:18:35Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:18:35Z,use_quota=True,user_id='b5012db78b0149b1aa7dcd2a71b660a7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:34.826757 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5bb00031-fada-4522-9a13-02d5742eebc0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:34.831497 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5bb00031-fada-4522-9a13-02d5742eebc0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:34.836960 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2bffb8f9-9c0a-4b74-af85-6b7f0c278cc7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:34.843493 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2bffb8f9-9c0a-4b74-af85-6b7f0c278cc7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd7631a2d-ca72-4741-b286-e081bbc5db7a', 'volume_name': 'volume-d7631a2d-ca72-4741-b286-e081bbc5db7a', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:34.848955 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1501fede-5b44-4da3-920e-d811ef301063) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:34.852985 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1501fede-5b44-4da3-920e-d811ef301063) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:34.858529 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fc130150-197b-401e-a4dc-b39fb853e3f8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:34.859777 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Volume d7631a2d-ca72-4741-b286-e081bbc5db7a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d7631a2d-ca72-4741-b286-e081bbc5db7a', 'volume_size': 1} Oct 01 21:18:34.860321 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] creating volume 'volume-d7631a2d-ca72-4741-b286-e081bbc5db7a' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:18:34.861274 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:34.929868 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:18:34.930484 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:18:34.935756 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fc130150-197b-401e-a4dc-b39fb853e3f8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d7631a2d-ca72-4741-b286-e081bbc5db7a', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:34.941418 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c31e8e76-60bf-4c94-b35f-b91f99dc1741) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:34.984915 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Volume volume-d7631a2d-ca72-4741-b286-e081bbc5db7a (d7631a2d-ca72-4741-b286-e081bbc5db7a): created successfully Oct 01 21:18:34.991319 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c31e8e76-60bf-4c94-b35f-b91f99dc1741) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:35.006705 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Flow 'volume_create_manager' (21078687-0435-4786-9f0e-6d7e2a1b4354) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:18:35.016808 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-15b6a86d-672a-472e-8f2f-62d0f50ee511 tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Created volume successfully. Oct 01 21:18:37.242051 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Acquiring lock "cinder-d7631a2d-ca72-4741-b286-e081bbc5db7a-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:18:37.264625 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Lock "cinder-d7631a2d-ca72-4741-b286-e081bbc5db7a-delete_volume" acquired by "delete_volume" :: waited 0.023s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:18:37.338195 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:37.422600 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:18:37.423079 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Volume volume-d7631a2d-ca72-4741-b286-e081bbc5db7a is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:18:37.434035 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] deleting rbd volume volume-d7631a2d-ca72-4741-b286-e081bbc5db7a {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:18:37.823393 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Created reservations ['7ef34f7d-69c7-4225-aaac-e9dd27d91444', 'e7ef2de1-ac4a-4478-a61e-f31cc2414759', '24477932-1f0c-4d60-95fe-152d029fe887', 'ce9bc312-cb8b-4acd-8935-b5c11b125c14'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:18:37.914570 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:37.975618 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:38.052971 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:18:38.062319 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Deleted volume successfully. Oct 01 21:18:38.091645 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6672eaf0-5c12-4cce-b05b-208e3a2bccea tempest-VolumeQuotasNegativeTestJSON-1033381038 None] Lock "cinder-d7631a2d-ca72-4741-b286-e081bbc5db7a-delete_volume" released by "delete_volume" :: held 0.827s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:18:45.950270 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=18,cinder:UPDATE=14,cinder:INSERT=2,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:18:45.970641 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:18:46.017048 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Flow 'volume_create_manager' (4ab8cc41-c6b3-411e-b796-0eda12ef6862) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:18:46.030174 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (454fa695-6908-46ab-8ba1-b6a8ba9ba094) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:46.069405 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (454fa695-6908-46ab-8ba1-b6a8ba9ba094) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:18:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesImageMetadata-Volume-148755273',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d3f7e49e1524439394e7a96d619c0c6d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:18:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:18:46Z,use_quota=True,user_id='33f3c126be294b1f8cd46f25d4087af0',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:46.076335 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f415efde-8b16-4d9f-bd8d-2cf7283e1941) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:46.082280 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f415efde-8b16-4d9f-bd8d-2cf7283e1941) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:46.089914 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5905ba81-8418-4a9e-9f9f-be4e96a70493) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:46.201511 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5905ba81-8418-4a9e-9f9f-be4e96a70493) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213', 'volume_name': 'volume-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:46.209536 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (92d74ac7-6216-4e83-be2b-31f1d1422915) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:46.214003 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (92d74ac7-6216-4e83-be2b-31f1d1422915) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:46.219394 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (04b66e29-1274-43fa-a9ff-be01a080f09c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:46.221552 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Volume e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:18:46.222454 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Cloning e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213 from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:18:46.223447 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:18:46.239161 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.016s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:18:46.258237 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 18, 46, 249493)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:18:46.259101 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:18:46.269256 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:18:46.274644 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.035s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:18:46.275712 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:18:46.285546 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 18, 46, 281544)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:18:46.286562 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:18:46.287118 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:18:46.304082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:46.379778 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] creating snapshot='volume-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:18:47.246357 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213.clone_snap' to 'volume-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:18:47.428674 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:18:47.430231 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:18:47.430690 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:18:47.448626 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:18:47.449617 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Marking volume e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:18:47.468052 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:18:47.468855 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Creating volume glance metadata for volume e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213 backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:18:47.549524 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (04b66e29-1274-43fa-a9ff-be01a080f09c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:47.559899 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f158df04-e6a8-4bb3-b0a0-b89904e805e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:47.577514 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Volume volume-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213 (e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213): created successfully Oct 01 21:18:47.581157 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f158df04-e6a8-4bb3-b0a0-b89904e805e5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:47.592485 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Flow 'volume_create_manager' (4ab8cc41-c6b3-411e-b796-0eda12ef6862) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:18:47.607916 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7523126b-e39c-4e79-8f1f-67fa5c6d72a0 tempest-VolumesImageMetadata-1358335890 None] Created volume successfully. Oct 01 21:18:48.533338 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] Acquiring lock "cinder-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:18:48.573459 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] Lock "cinder-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213-delete_volume" acquired by "delete_volume" :: waited 0.038s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:18:48.646683 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:48.766009 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:18:48.782127 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] deleting rbd volume volume-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:18:49.054450 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:18:49.087483 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] deleting parent snapshot volume-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:18:49.341051 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] Created reservations ['54af5892-c2f2-48f8-96d0-78d153de80c8', '084078e7-be6b-4a72-8a07-84a48075b87b', 'bbf54485-665a-47f4-a558-1d12ac2e57da', 'c536fa89-668c-41b4-8f18-2b97f8385506'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:18:49.412535 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:49.463103 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:49.532152 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:18:49.538378 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] Deleted volume successfully. Oct 01 21:18:49.557713 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fa13bd54-e3b2-41ca-bd1e-b5f173a09143 tempest-VolumesImageMetadata-1358335890 None] Lock "cinder-e2c8a7b8-f474-4b5a-a7c4-93c4e20c7213-delete_volume" released by "delete_volume" :: held 0.984s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:18:58.298139 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:18:58.351424 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Flow 'volume_create_manager' (1a9d18ec-75ca-4f8a-8249-0bc44778b656) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:18:58.362710 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (64a5ce68-f889-413c-82d1-689ccb3621ea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:58.393246 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (64a5ce68-f889-413c-82d1-689ccb3621ea) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:18:58Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AbsoluteLimitsTests-Volume-1917660408',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='1e8b95027abe453085cf757cc0b64874',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:18:58Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:18:58Z,use_quota=True,user_id='31d66b5c448f4914a1bb2a58715b699d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:58.400207 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1092d7cb-ca5d-443a-b5b1-d58525fe4a23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:58.409220 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1092d7cb-ca5d-443a-b5b1-d58525fe4a23) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:58.426131 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7aaac76a-583e-4e18-a6b3-24aafdb76cbe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:58.439370 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7aaac76a-583e-4e18-a6b3-24aafdb76cbe) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8', 'volume_name': 'volume-fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:58.444575 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c3d5da9b-d038-47dd-88e7-1bfeb9b4326f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:58.449974 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c3d5da9b-d038-47dd-88e7-1bfeb9b4326f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:58.455816 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8e9b2d7c-3d33-40c8-b1d6-d9c475a863f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:58.457982 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Volume fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8', 'volume_size': 1} Oct 01 21:18:58.458797 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] creating volume 'volume-fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:18:58.459762 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:58.600175 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:18:58.600852 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:18:58.606462 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8e9b2d7c-3d33-40c8-b1d6-d9c475a863f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:58.616958 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2ab8fe59-1952-4fd9-b69f-d112c4e8c920) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:18:58.643641 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Volume volume-fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8 (fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8): created successfully Oct 01 21:18:58.649819 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2ab8fe59-1952-4fd9-b69f-d112c4e8c920) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:18:58.655287 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Flow 'volume_create_manager' (1a9d18ec-75ca-4f8a-8249-0bc44778b656) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:18:58.690035 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f4a050dd-7d2b-4c22-94fb-8a360b2db12f tempest-AbsoluteLimitsTests-40940289 None] Created volume successfully. Oct 01 21:18:59.709883 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] Acquiring lock "cinder-fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:18:59.724923 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] Lock "cinder-fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8-delete_volume" acquired by "delete_volume" :: waited 0.017s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:18:59.786433 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:18:59.849304 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:18:59.849925 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] Volume volume-fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:18:59.859551 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] deleting rbd volume volume-fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:19:00.202354 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=66,cinder:UPDATE=21,cinder:DELETE=2,cinder:INSERT=11 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:00.279645 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] Created reservations ['a4988482-5f39-4e7a-8c89-14be57fb47f3', 'e3a12e5d-3df7-417b-93c1-a37a173c6ec2', '83798a61-ff5f-4dba-951a-38613e9517e4', 'bccbc9cc-6593-4296-a01c-71ab28615eba'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:19:00.359258 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:19:00.432299 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:19:00.485395 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:19:00.492954 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] Deleted volume successfully. Oct 01 21:19:00.517057 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9f69a70d-1907-497c-9c2d-d02a27454ec8 tempest-AbsoluteLimitsTests-40940289 None] Lock "cinder-fba9e8d0-11b0-4456-8fa8-17fbcc95c3d8-delete_volume" released by "delete_volume" :: held 0.792s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:19:06.591982 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:19:06.635172 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Flow 'volume_create_manager' (ca2f4ff7-5d38-495c-92c7-4ff7d6f4d88c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:19:06.644689 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a759da08-6791-4c4b-8ed8-bdb65345d75f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:19:06.670126 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a759da08-6791-4c4b-8ed8-bdb65345d75f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:19:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-126130459',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cd42b161-77e3-4505-a383-4323014595bf,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5f12200ca5c649cd993499b7cb7e382f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:19:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:19:07Z,use_quota=True,user_id='ecdf7bba776f4b9c94f2653679505303',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:19:06.678735 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4090d241-c32a-4555-b254-3b038f7a7256) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:19:06.684553 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4090d241-c32a-4555-b254-3b038f7a7256) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:19:06.691166 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bf98a850-9ce6-4845-afe0-43ad813aa7ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:19:06.695516 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bf98a850-9ce6-4845-afe0-43ad813aa7ff) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'cd42b161-77e3-4505-a383-4323014595bf', 'volume_name': 'volume-cd42b161-77e3-4505-a383-4323014595bf', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:19:06.701345 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (96c1f49d-7f4b-45b9-af49-322ad2312ce9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:19:06.707460 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (96c1f49d-7f4b-45b9-af49-322ad2312ce9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:19:06.714305 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bc56b6cc-3a5c-4170-b527-48d31a600561) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:19:06.716163 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Volume cd42b161-77e3-4505-a383-4323014595bf: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-cd42b161-77e3-4505-a383-4323014595bf', 'volume_size': 1} Oct 01 21:19:06.716948 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] creating volume 'volume-cd42b161-77e3-4505-a383-4323014595bf' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:19:06.718056 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:19:06.849366 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:19:06.850001 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:19:06.856428 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bc56b6cc-3a5c-4170-b527-48d31a600561) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cd42b161-77e3-4505-a383-4323014595bf', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:19:06.865822 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a0ff3d5d-1ccd-4a44-ad3f-78151c7679a6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:19:06.887919 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Volume volume-cd42b161-77e3-4505-a383-4323014595bf (cd42b161-77e3-4505-a383-4323014595bf): created successfully Oct 01 21:19:06.892335 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a0ff3d5d-1ccd-4a44-ad3f-78151c7679a6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:19:06.898407 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Flow 'volume_create_manager' (ca2f4ff7-5d38-495c-92c7-4ff7d6f4d88c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:19:06.909987 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4228b8b1-d159-451d-a255-c2e03fd97815 tempest-VolumesActionsTest-1307507385 None] Created volume successfully. Oct 01 21:19:16.918186 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:INSERT=2,cinder:SELECT=10,cinder:UPDATE=13,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:23.820009 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:19:23.820009 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:19:23.911034 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:19:24.003231 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:19:24.362655 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=21,cinder:SELECT=49,cinder:INSERT=9,cinder:DELETE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:25.568624 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7e7f8394-4b86-4053-9891-8f6e6af748c9 tempest-VolumesActionsTest-1307507385 None] Acquiring lock "cinder-cd42b161-77e3-4505-a383-4323014595bf" by "attach_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:19:25.581579 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7e7f8394-4b86-4053-9891-8f6e6af748c9 tempest-VolumesActionsTest-1307507385 None] Lock "cinder-cd42b161-77e3-4505-a383-4323014595bf" acquired by "attach_volume" :: waited 0.013s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:19:25.660711 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7e7f8394-4b86-4053-9891-8f6e6af748c9 tempest-VolumesActionsTest-1307507385 None] Attaching volume cd42b161-77e3-4505-a383-4323014595bf to instance 490563c9-eb63-4ef4-beb8-f9f7e560fdf1 at mountpoint /dev/sdb on host None. Oct 01 21:19:25.729961 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7e7f8394-4b86-4053-9891-8f6e6af748c9 tempest-VolumesActionsTest-1307507385 None] Attach volume completed successfully. Oct 01 21:19:25.742747 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7e7f8394-4b86-4053-9891-8f6e6af748c9 tempest-VolumesActionsTest-1307507385 None] Lock "cinder-cd42b161-77e3-4505-a383-4323014595bf" released by "attach_volume" :: held 0.161s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:19:25.967737 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f3e1b018-8b94-4a99-9046-c7bbd15f5cde tempest-VolumesActionsTest-1307507385 None] Acquiring lock "cinder-cd42b161-77e3-4505-a383-4323014595bf-detach_volume" by "detach_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:19:25.995595 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f3e1b018-8b94-4a99-9046-c7bbd15f5cde tempest-VolumesActionsTest-1307507385 None] Lock "cinder-cd42b161-77e3-4505-a383-4323014595bf-detach_volume" acquired by "detach_volume" :: waited 0.027s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:19:25.996441 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f3e1b018-8b94-4a99-9046-c7bbd15f5cde tempest-VolumesActionsTest-1307507385 None] Detaching volume cd42b161-77e3-4505-a383-4323014595bf from instance 490563c9-eb63-4ef4-beb8-f9f7e560fdf1. Oct 01 21:19:26.109395 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f3e1b018-8b94-4a99-9046-c7bbd15f5cde tempest-VolumesActionsTest-1307507385 None] Detach volume completed successfully. Oct 01 21:19:26.115231 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f3e1b018-8b94-4a99-9046-c7bbd15f5cde tempest-VolumesActionsTest-1307507385 None] Lock "cinder-cd42b161-77e3-4505-a383-4323014595bf-detach_volume" released by "detach_volume" :: held 0.120s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:19:36.107671 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=15,cinder:INSERT=1,cinder:UPDATE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:47.628258 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-81dc0978-e88c-4aec-856d-fe9c6c957d55 tempest-VolumesActionsTest-1307507385 None] Acquiring lock "cinder-cd42b161-77e3-4505-a383-4323014595bf" by "attach_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:19:47.645888 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-81dc0978-e88c-4aec-856d-fe9c6c957d55 tempest-VolumesActionsTest-1307507385 None] Lock "cinder-cd42b161-77e3-4505-a383-4323014595bf" acquired by "attach_volume" :: waited 0.018s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:19:47.679210 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=15,cinder:UPDATE=4,cinder:INSERT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:19:47.743719 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-81dc0978-e88c-4aec-856d-fe9c6c957d55 tempest-VolumesActionsTest-1307507385 None] Attaching volume cd42b161-77e3-4505-a383-4323014595bf to instance 0f8da90f-c891-41ed-8e01-153fef78d0b1 at mountpoint /dev/sdb on host None. Oct 01 21:19:47.801199 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-81dc0978-e88c-4aec-856d-fe9c6c957d55 tempest-VolumesActionsTest-1307507385 None] Attach volume completed successfully. Oct 01 21:19:47.812864 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-81dc0978-e88c-4aec-856d-fe9c6c957d55 tempest-VolumesActionsTest-1307507385 None] Lock "cinder-cd42b161-77e3-4505-a383-4323014595bf" released by "attach_volume" :: held 0.167s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:19:48.135022 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-35650ca5-6ce7-4dc7-b93d-b9e03a2eb5cb tempest-VolumesActionsTest-1307507385 None] Acquiring lock "cinder-cd42b161-77e3-4505-a383-4323014595bf-detach_volume" by "detach_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:19:48.149492 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-35650ca5-6ce7-4dc7-b93d-b9e03a2eb5cb tempest-VolumesActionsTest-1307507385 None] Lock "cinder-cd42b161-77e3-4505-a383-4323014595bf-detach_volume" acquired by "detach_volume" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:19:48.155120 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-35650ca5-6ce7-4dc7-b93d-b9e03a2eb5cb tempest-VolumesActionsTest-1307507385 None] Detaching volume cd42b161-77e3-4505-a383-4323014595bf from instance 0f8da90f-c891-41ed-8e01-153fef78d0b1. Oct 01 21:19:48.250291 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-35650ca5-6ce7-4dc7-b93d-b9e03a2eb5cb tempest-VolumesActionsTest-1307507385 None] Detach volume completed successfully. Oct 01 21:19:48.263569 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-35650ca5-6ce7-4dc7-b93d-b9e03a2eb5cb tempest-VolumesActionsTest-1307507385 None] Lock "cinder-cd42b161-77e3-4505-a383-4323014595bf-detach_volume" released by "detach_volume" :: held 0.114s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:19:55.213868 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-d9ca51b7-90e9-4d8d-b603-9e69fb78db33 tempest-VolumesActionsTest-1307507385 None] Running cmd (subprocess): rbd export --pool volumes volume-cd42b161-77e3-4505-a383-4323014595bf /opt/stack/data/cinder/conversion/volume-cd42b161-77e3-4505-a383-4323014595bf-bb25de42-01ec-47d6-b327-259cc3ab373b --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:19:56.988555 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-d9ca51b7-90e9-4d8d-b603-9e69fb78db33 tempest-VolumesActionsTest-1307507385 None] CMD "rbd export --pool volumes volume-cd42b161-77e3-4505-a383-4323014595bf /opt/stack/data/cinder/conversion/volume-cd42b161-77e3-4505-a383-4323014595bf-bb25de42-01ec-47d6-b327-259cc3ab373b --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.769s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:19:57.010908 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.image_utils [None req-d9ca51b7-90e9-4d8d-b603-9e69fb78db33 tempest-VolumesActionsTest-1307507385 None] bb25de42-01ec-47d6-b327-259cc3ab373b was raw, no need to convert to raw {{(pid=133391) upload_volume /opt/stack/cinder/cinder/image/image_utils.py:851}} Oct 01 21:20:14.368885 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=23,cinder:UPDATE=6,cinder:INSERT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:23.816062 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:20:23.817964 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:20:23.917049 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:20:23.969189 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:20:24.365698 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=14,cinder:UPDATE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:20:39.444485 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9ca51b7-90e9-4d8d-b603-9e69fb78db33 tempest-VolumesActionsTest-1307507385 None] Uploaded volume to glance image-id: bb25de42-01ec-47d6-b327-259cc3ab373b. {{(pid=133391) copy_volume_to_image /opt/stack/cinder/cinder/volume/manager.py:1791}} Oct 01 21:20:39.466106 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d9ca51b7-90e9-4d8d-b603-9e69fb78db33 tempest-VolumesActionsTest-1307507385 None] Copy volume to image completed successfully. Oct 01 21:20:40.344142 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] Acquiring lock "cinder-cd42b161-77e3-4505-a383-4323014595bf-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:20:40.374940 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] Lock "cinder-cd42b161-77e3-4505-a383-4323014595bf-delete_volume" acquired by "delete_volume" :: waited 0.031s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:20:40.483106 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:20:40.544978 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:20:40.545698 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] Volume volume-cd42b161-77e3-4505-a383-4323014595bf is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:20:40.560173 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] deleting rbd volume volume-cd42b161-77e3-4505-a383-4323014595bf {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:20:41.056560 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] Created reservations ['d35030aa-8233-49aa-b54b-e2878da3808f', 'ea405e06-a3aa-4686-b4b5-066bea19eca4', '1ce97c12-f8f3-43ec-9c0d-2fe7d4bf2168', '4365ee57-70e0-4d48-8edd-b89649271f87'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:20:41.161421 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:20:41.211286 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:20:41.271881 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:20:41.284675 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] Deleted volume successfully. Oct 01 21:20:41.309525 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f9acc2a3-ec79-4047-b74e-d8bec393ed6d tempest-VolumesActionsTest-1307507385 None] Lock "cinder-cd42b161-77e3-4505-a383-4323014595bf-delete_volume" released by "delete_volume" :: held 0.934s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:20:54.389363 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=23,cinder:UPDATE=8,cinder:INSERT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:14.374180 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=17,cinder:UPDATE=13,cinder:INSERT=2,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:14.610913 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:21:14.666822 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (4a88994a-791b-4ae9-afa8-b0dec8e0bf75) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:14.683076 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (44516af8-5fa1-487c-a460-42166507bc30) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:14.735412 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (44516af8-5fa1-487c-a460-42166507bc30) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:21:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-239182267',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f4acfc21-c684-406a-9013-b6774bfb4f6a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='df0e56b020444dd28af868590b1bccf4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:21:15Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:21:15Z,use_quota=True,user_id='eb0bf5fce885439eaa16a0871c2992ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:14.765323 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (48d07706-b3bd-438e-89a8-7f58a0872f23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:14.784312 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (48d07706-b3bd-438e-89a8-7f58a0872f23) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:14.805966 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (111951a8-ac3f-4a8b-bcc0-6600011f1fd4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:15.009468 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (111951a8-ac3f-4a8b-bcc0-6600011f1fd4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'f4acfc21-c684-406a-9013-b6774bfb4f6a', 'volume_name': 'volume-f4acfc21-c684-406a-9013-b6774bfb4f6a', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:15.016873 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7d431a3f-de7c-4dce-bd12-295365a9a787) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:15.021817 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7d431a3f-de7c-4dce-bd12-295365a9a787) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:15.050616 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0919e7d0-cbe4-471b-95bc-865ee786b9d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:15.053656 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Volume f4acfc21-c684-406a-9013-b6774bfb4f6a: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-f4acfc21-c684-406a-9013-b6774bfb4f6a', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:21:15.054656 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Cloning f4acfc21-c684-406a-9013-b6774bfb4f6a from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:21:15.055662 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:21:15.072012 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:21:15.109942 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 21, 15, 96013)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:21:15.110997 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:21:15.111570 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:21:15.121226 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.050s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:21:15.122479 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:21:15.135881 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 21, 15, 130250)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:21:15.138303 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:21:15.138749 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:21:15.176144 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:15.289203 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] creating snapshot='volume-f4acfc21-c684-406a-9013-b6774bfb4f6a.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:21:16.111851 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-f4acfc21-c684-406a-9013-b6774bfb4f6a.clone_snap' to 'volume-f4acfc21-c684-406a-9013-b6774bfb4f6a' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:21:16.316124 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:21:16.318928 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:21:16.319426 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:21:16.359390 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:21:16.360410 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Marking volume f4acfc21-c684-406a-9013-b6774bfb4f6a as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:21:16.375245 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to f4acfc21-c684-406a-9013-b6774bfb4f6a. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:21:16.375679 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Creating volume glance metadata for volume f4acfc21-c684-406a-9013-b6774bfb4f6a backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:21:16.496670 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0919e7d0-cbe4-471b-95bc-865ee786b9d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f4acfc21-c684-406a-9013-b6774bfb4f6a', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:16.508759 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ec3ebf1a-ae75-41c8-92d1-3b9722e70c91) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:16.535179 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Volume volume-f4acfc21-c684-406a-9013-b6774bfb4f6a (f4acfc21-c684-406a-9013-b6774bfb4f6a): created successfully Oct 01 21:21:16.541155 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ec3ebf1a-ae75-41c8-92d1-3b9722e70c91) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:16.549189 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (4a88994a-791b-4ae9-afa8-b0dec8e0bf75) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:16.566989 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6eeb7564-6502-4c9c-9c96-8d1d1746e036 tempest-TestVolumeBootPattern-1545759599 None] Created volume successfully. Oct 01 21:21:17.167449 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6d50849a-57b1-4aab-8a0d-a2e35277521f tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:18.300818 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6d50849a-57b1-4aab-8a0d-a2e35277521f tempest-TestVolumeBootPattern-1545759599 None] Create snapshot completed successfully Oct 01 21:21:23.075411 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:21:23.144587 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (33f10939-f08a-421a-88f3-eedc42a97755) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:23.151882 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4f7bdf5d-a053-4f84-85fe-790f8dc22e90) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:23.194275 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4f7bdf5d-a053-4f84-85fe-790f8dc22e90) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:21:23Z,deleted=False,deleted_at=None,display_description='',display_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2af50836-f380-4890-85b3-c11cb6aac060,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='df0e56b020444dd28af868590b1bccf4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:21:23Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=c73daf27-b1ba-44f2-8829-07b97df7622f,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:21:23Z,use_quota=True,user_id='eb0bf5fce885439eaa16a0871c2992ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:23.208237 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (637a5540-ba03-4c8b-a322-c1846fe3594d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:23.219696 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (637a5540-ba03-4c8b-a322-c1846fe3594d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:23.225400 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (59fc1794-849f-4f53-81bd-308db6662245) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:23.240570 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (59fc1794-849f-4f53-81bd-308db6662245) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '2af50836-f380-4890-85b3-c11cb6aac060', 'volume_name': 'volume-2af50836-f380-4890-85b3-c11cb6aac060', 'volume_size': 1, 'snapshot_id': 'c73daf27-b1ba-44f2-8829-07b97df7622f'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:23.255374 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a451af71-bd0e-48a7-8264-b4eecbff9640) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:23.260068 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a451af71-bd0e-48a7-8264-b4eecbff9640) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:23.277694 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4838b144-3a83-4225-b789-4aa11f1b258b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:23.279127 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Volume 2af50836-f380-4890-85b3-c11cb6aac060: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-2af50836-f380-4890-85b3-c11cb6aac060', 'volume_size': 1, 'snapshot_id': 'c73daf27-b1ba-44f2-8829-07b97df7622f'} Oct 01 21:21:23.310287 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] cloning volumes/volume-f4acfc21-c684-406a-9013-b6774bfb4f6a@snapshot-c73daf27-b1ba-44f2-8829-07b97df7622f to volume-2af50836-f380-4890-85b3-c11cb6aac060 {{(pid=133391) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Oct 01 21:21:23.311583 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:23.424687 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:23.649863 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:21:23.650640 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:21:23.700068 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Marking volume 2af50836-f380-4890-85b3-c11cb6aac060 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:21:23.720183 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Copying metadata from snapshot c73daf27-b1ba-44f2-8829-07b97df7622f to 2af50836-f380-4890-85b3-c11cb6aac060. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:431}} Oct 01 21:21:23.816293 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:21:23.819343 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:23.924501 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:24.003823 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:21:24.034642 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4838b144-3a83-4225-b789-4aa11f1b258b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2af50836-f380-4890-85b3-c11cb6aac060', 'volume_size': 1, 'snapshot_id': 'c73daf27-b1ba-44f2-8829-07b97df7622f'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:24.048283 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3919fba4-3e8d-40e4-8d8b-531ca8f9b3d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:24.067389 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Volume volume-2af50836-f380-4890-85b3-c11cb6aac060 (2af50836-f380-4890-85b3-c11cb6aac060): created successfully Oct 01 21:21:24.074831 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3919fba4-3e8d-40e4-8d8b-531ca8f9b3d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:24.080568 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (33f10939-f08a-421a-88f3-eedc42a97755) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:24.112004 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-661b1e7f-67b2-41bb-a943-f0b1ffd03d11 tempest-TestVolumeBootPattern-1545759599 None] Created volume successfully. Oct 01 21:21:26.636597 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=44,cinder:UPDATE=9,cinder:INSERT=18,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:26.655854 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-dc06f2a7-71d1-4e49-aa46-b703954c7543 tempest-TestVolumeBootPattern-1545759599 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:21:28.015981 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-dc06f2a7-71d1-4e49-aa46-b703954c7543 tempest-TestVolumeBootPattern-1545759599 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.359s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:21:28.017561 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-dc06f2a7-71d1-4e49-aa46-b703954c7543 tempest-TestVolumeBootPattern-1545759599 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2af50836-f380-4890-85b3-c11cb6aac060', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '2af50836-f380-4890-85b3-c11cb6aac060', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:21:28.070378 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-dc06f2a7-71d1-4e49-aa46-b703954c7543 tempest-TestVolumeBootPattern-1545759599 None] Connection info returned from driver {'name': 'volumes/volume-2af50836-f380-4890-85b3-c11cb6aac060', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '2af50836-f380-4890-85b3-c11cb6aac060', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '810a9acd-4cee-4d6f-b0e9-13ef2a5570ed'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:21:28.203281 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-4fe59254-eedb-46de-9e65-053abd9d3939 req-dc06f2a7-71d1-4e49-aa46-b703954c7543 tempest-TestVolumeBootPattern-1545759599 None] attachment_update completed successfully. Oct 01 21:21:31.378377 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:21:31.404862 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Flow 'volume_create_manager' (b8cf9a4e-7d79-4c9c-af9e-511a09b50df8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:31.413448 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8e61a04b-7992-404f-9b73-48497aecdbfe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:31.432317 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8e61a04b-7992-404f-9b73-48497aecdbfe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:21:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBackupRestore-volume-1179375556',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2c3c1619-9d87-4421-83b1-88c4fc46ebc4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='90cb48440cb446ea9abfc00ed0f53779',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:21:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:21:31Z,use_quota=True,user_id='55dadd491ec1441aac4171eff46bd679',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:31.438091 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0f88e22f-7a4b-425b-a4ad-7c9ea42dc028) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:31.442291 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0f88e22f-7a4b-425b-a4ad-7c9ea42dc028) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:31.448018 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5f0997d7-ca9a-4ebb-8d1a-35555773f542) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:31.567912 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5f0997d7-ca9a-4ebb-8d1a-35555773f542) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '2c3c1619-9d87-4421-83b1-88c4fc46ebc4', 'volume_name': 'volume-2c3c1619-9d87-4421-83b1-88c4fc46ebc4', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:31.576210 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b69720b4-d18a-44b8-9a48-68bf7c6dd264) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:31.580760 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b69720b4-d18a-44b8-9a48-68bf7c6dd264) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:31.586215 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (87b66593-234d-4165-9225-86e7e21ae266) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:31.587743 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Volume 2c3c1619-9d87-4421-83b1-88c4fc46ebc4: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-2c3c1619-9d87-4421-83b1-88c4fc46ebc4', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:21:31.588516 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Cloning 2c3c1619-9d87-4421-83b1-88c4fc46ebc4 from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:21:31.589446 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:21:31.598573 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:21:31.611764 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 21, 31, 604179)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:21:31.612810 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:21:31.613330 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:21:31.620663 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.022s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:21:31.621523 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:21:31.630785 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 21, 31, 626689)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:21:31.631547 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:21:31.632461 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:21:31.647140 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:31.753272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] creating snapshot='volume-2c3c1619-9d87-4421-83b1-88c4fc46ebc4.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:21:32.383355 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-2c3c1619-9d87-4421-83b1-88c4fc46ebc4.clone_snap' to 'volume-2c3c1619-9d87-4421-83b1-88c4fc46ebc4' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:21:32.613085 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:21:32.614757 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:21:32.615723 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:21:32.637205 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:21:32.639210 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Marking volume 2c3c1619-9d87-4421-83b1-88c4fc46ebc4 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:21:32.664676 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to 2c3c1619-9d87-4421-83b1-88c4fc46ebc4. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:21:32.664676 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Creating volume glance metadata for volume 2c3c1619-9d87-4421-83b1-88c4fc46ebc4 backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:21:32.736625 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (87b66593-234d-4165-9225-86e7e21ae266) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2c3c1619-9d87-4421-83b1-88c4fc46ebc4', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:32.741691 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a98eca59-533d-4691-9283-df4ec280cd91) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:32.752851 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Volume volume-2c3c1619-9d87-4421-83b1-88c4fc46ebc4 (2c3c1619-9d87-4421-83b1-88c4fc46ebc4): created successfully Oct 01 21:21:32.757232 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a98eca59-533d-4691-9283-df4ec280cd91) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:32.762557 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Flow 'volume_create_manager' (b8cf9a4e-7d79-4c9c-af9e-511a09b50df8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:32.770742 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e2ba43d2-f92f-443b-b279-cac21b05225a tempest-TestVolumeBackupRestore-262965167 None] Created volume successfully. Oct 01 21:21:33.868965 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ee0ae22f-867c-4cbc-a07c-daa910932050 tempest-TestVolumeBackupRestore-262965167 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-10-01T21:21:33Z,data_timestamp=2022-10-01T21:21:33Z,deleted=False,deleted_at=None,display_description=None,display_name=None,encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=eede824b-1fe2-4998-8fdf-84a59ace0173,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='90cb48440cb446ea9abfc00ed0f53779',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='55dadd491ec1441aac4171eff46bd679',volume_id=2c3c1619-9d87-4421-83b1-88c4fc46ebc4) Oct 01 21:21:33.871804 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.backup.rpcapi [None req-ee0ae22f-867c-4cbc-a07c-daa910932050 tempest-TestVolumeBackupRestore-262965167 None] continue_backup in rpcapi backup_id eede824b-1fe2-4998-8fdf-84a59ace0173 {{(pid=133391) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Oct 01 21:21:34.118077 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-ee0ae22f-867c-4cbc-a07c-daa910932050 tempest-TestVolumeBackupRestore-262965167 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:21:34.947397 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-ee0ae22f-867c-4cbc-a07c-daa910932050 tempest-TestVolumeBackupRestore-262965167 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.824s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:21:34.947397 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ee0ae22f-867c-4cbc-a07c-daa910932050 tempest-TestVolumeBackupRestore-262965167 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2c3c1619-9d87-4421-83b1-88c4fc46ebc4', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '2c3c1619-9d87-4421-83b1-88c4fc46ebc4', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:21:34.969871 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ee0ae22f-867c-4cbc-a07c-daa910932050 tempest-TestVolumeBackupRestore-262965167 None] Initialize volume connection completed successfully. Oct 01 21:21:36.735011 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:21:36.760415 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Flow 'volume_create_manager' (b716fbfd-d85a-4450-aad4-c156807fc71c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:36.767658 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a115593f-932a-4f0e-b23e-e8aac8707340) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:36.773161 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=62,cinder:UPDATE=11,cinder:INSERT=25,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:36.786031 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a115593f-932a-4f0e-b23e-e8aac8707340) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:21:36Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Volume-1553982542',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=7879d8cf-be58-4353-baa6-ec93c3a8634a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='cca952fbe7c84efda0ecaf0dfb56a658',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:21:37Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:21:37Z,use_quota=True,user_id='733cf56b323f4844b6578ea1f9a46cfb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:36.799398 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (17fc9a8e-1e64-4fec-8fed-e3616e3beea4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:36.807292 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (17fc9a8e-1e64-4fec-8fed-e3616e3beea4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:36.813171 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f7de7830-652f-4119-b3a0-40f041fc6d4f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:36.817171 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f7de7830-652f-4119-b3a0-40f041fc6d4f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '7879d8cf-be58-4353-baa6-ec93c3a8634a', 'volume_name': 'volume-7879d8cf-be58-4353-baa6-ec93c3a8634a', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:36.822296 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e7588885-9fd2-4d91-90ab-43da33117e3f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:36.826228 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e7588885-9fd2-4d91-90ab-43da33117e3f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:36.830893 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0d9c3aa9-b412-4a67-804b-3022c772a34b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:36.831992 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Volume 7879d8cf-be58-4353-baa6-ec93c3a8634a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-7879d8cf-be58-4353-baa6-ec93c3a8634a', 'volume_size': 1} Oct 01 21:21:36.832613 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] creating volume 'volume-7879d8cf-be58-4353-baa6-ec93c3a8634a' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:21:36.833769 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:37.524561 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:21:37.524908 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:21:37.528719 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0d9c3aa9-b412-4a67-804b-3022c772a34b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7879d8cf-be58-4353-baa6-ec93c3a8634a', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:37.533653 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (67e019fb-b2f4-4cea-ba5c-6f5675a64295) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:37.553027 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Volume volume-7879d8cf-be58-4353-baa6-ec93c3a8634a (7879d8cf-be58-4353-baa6-ec93c3a8634a): created successfully Oct 01 21:21:37.557194 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (67e019fb-b2f4-4cea-ba5c-6f5675a64295) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:37.563281 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Flow 'volume_create_manager' (b716fbfd-d85a-4450-aad4-c156807fc71c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:37.576306 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-629bbb2b-0fb9-4775-9352-2d7b893ed048 tempest-VolumesBackupsAdminTest-426242016 None] Created volume successfully. Oct 01 21:21:37.999714 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5da086cf-8174-4ecf-b876-80ff5952a17e tempest-VolumesBackupsAdminTest-426242016 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-10-01T21:21:37Z,data_timestamp=2022-10-01T21:21:37Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-2091089304',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=26db3943-326a-45bf-8c90-6703318597d3,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='cca952fbe7c84efda0ecaf0dfb56a658',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='733cf56b323f4844b6578ea1f9a46cfb',volume_id=7879d8cf-be58-4353-baa6-ec93c3a8634a) Oct 01 21:21:38.000499 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.backup.rpcapi [None req-5da086cf-8174-4ecf-b876-80ff5952a17e tempest-VolumesBackupsAdminTest-426242016 None] continue_backup in rpcapi backup_id 26db3943-326a-45bf-8c90-6703318597d3 {{(pid=133391) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Oct 01 21:21:38.111756 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-5da086cf-8174-4ecf-b876-80ff5952a17e tempest-VolumesBackupsAdminTest-426242016 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:21:38.947515 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-5da086cf-8174-4ecf-b876-80ff5952a17e tempest-VolumesBackupsAdminTest-426242016 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.835s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:21:38.948832 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5da086cf-8174-4ecf-b876-80ff5952a17e tempest-VolumesBackupsAdminTest-426242016 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-7879d8cf-be58-4353-baa6-ec93c3a8634a', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '7879d8cf-be58-4353-baa6-ec93c3a8634a', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:21:39.007328 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5da086cf-8174-4ecf-b876-80ff5952a17e tempest-VolumesBackupsAdminTest-426242016 None] Initialize volume connection completed successfully. Oct 01 21:21:39.477300 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-46eadc45-76bb-4cb4-b3c4-c9763b79bad9 tempest-TestVolumeBootPattern-1545759599 None] Terminate volume connection completed successfully. Oct 01 21:21:39.478083 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-46eadc45-76bb-4cb4-b3c4-c9763b79bad9 tempest-TestVolumeBootPattern-1545759599 None] Deleting attachment 810a9acd-4cee-4d6f-b0e9-13ef2a5570ed. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:21:39.645610 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-ff99bd6c-6d5a-42b4-94cf-3a8e672e70bf tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-2af50836-f380-4890-85b3-c11cb6aac060-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:21:39.676771 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-ff99bd6c-6d5a-42b4-94cf-3a8e672e70bf tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-2af50836-f380-4890-85b3-c11cb6aac060-delete_volume" acquired by "delete_volume" :: waited 0.031s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:21:39.744102 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-ff99bd6c-6d5a-42b4-94cf-3a8e672e70bf tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:40.179815 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-ff99bd6c-6d5a-42b4-94cf-3a8e672e70bf tempest-TestVolumeBootPattern-1545759599 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:21:40.377621 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-ff99bd6c-6d5a-42b4-94cf-3a8e672e70bf tempest-TestVolumeBootPattern-1545759599 None] deleting rbd volume volume-2af50836-f380-4890-85b3-c11cb6aac060 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:21:41.798277 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-ff99bd6c-6d5a-42b4-94cf-3a8e672e70bf tempest-TestVolumeBootPattern-1545759599 None] Created reservations ['2fee4aa7-64cc-4980-95b0-f2e2bd2fb157', '8957f30e-0dfe-4726-8408-7a7337e3cec1', 'c7d18ac8-8c70-4875-a7aa-91f98eeb3f20', '48577943-6694-4e69-8fcd-e4d9177c81d6'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:21:41.861875 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-ff99bd6c-6d5a-42b4-94cf-3a8e672e70bf tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:41.918628 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-ff99bd6c-6d5a-42b4-94cf-3a8e672e70bf tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:41.977877 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-ff99bd6c-6d5a-42b4-94cf-3a8e672e70bf tempest-TestVolumeBootPattern-1545759599 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:21:41.986297 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-ff99bd6c-6d5a-42b4-94cf-3a8e672e70bf tempest-TestVolumeBootPattern-1545759599 None] Deleted volume successfully. Oct 01 21:21:42.005490 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-9756edda-2377-4e11-ac40-c5b2e1c32831 req-ff99bd6c-6d5a-42b4-94cf-3a8e672e70bf tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-2af50836-f380-4890-85b3-c11cb6aac060-delete_volume" released by "delete_volume" :: held 2.328s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:21:42.118711 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e93441c1-7bbb-40d3-b986-7ed8aea6d29e tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-c73daf27-b1ba-44f2-8829-07b97df7622f-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:21:42.130778 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e93441c1-7bbb-40d3-b986-7ed8aea6d29e tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-c73daf27-b1ba-44f2-8829-07b97df7622f-delete_snapshot" acquired by "delete_snapshot" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:21:42.132141 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e93441c1-7bbb-40d3-b986-7ed8aea6d29e tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:42.225611 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5da086cf-8174-4ecf-b876-80ff5952a17e tempest-VolumesBackupsAdminTest-426242016 None] Terminate volume connection completed successfully. Oct 01 21:21:42.245166 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5da086cf-8174-4ecf-b876-80ff5952a17e tempest-VolumesBackupsAdminTest-426242016 None] Remove volume export completed successfully. Oct 01 21:21:43.154068 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-e93441c1-7bbb-40d3-b986-7ed8aea6d29e tempest-TestVolumeBootPattern-1545759599 None] Created reservations ['76332515-4f0f-4981-9515-636bce0ec176', '35f8f77b-87ec-4660-8b9f-9db9e68781b7', '77ca8dc4-f481-4aaa-a23c-449c04a704d9', '2126d926-51f4-4f2e-92d7-4fbc26307dcb'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:21:43.227323 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e93441c1-7bbb-40d3-b986-7ed8aea6d29e tempest-TestVolumeBootPattern-1545759599 None] Delete snapshot completed successfully. Oct 01 21:21:43.236532 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e93441c1-7bbb-40d3-b986-7ed8aea6d29e tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-c73daf27-b1ba-44f2-8829-07b97df7622f-delete_snapshot" released by "delete_snapshot" :: held 1.107s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:21:43.248404 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ee0ae22f-867c-4cbc-a07c-daa910932050 tempest-TestVolumeBackupRestore-262965167 None] Terminate volume connection completed successfully. Oct 01 21:21:43.270681 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ee0ae22f-867c-4cbc-a07c-daa910932050 tempest-TestVolumeBackupRestore-262965167 None] Remove volume export completed successfully. Oct 01 21:21:44.276958 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-f4acfc21-c684-406a-9013-b6774bfb4f6a-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:21:44.285634 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-f4acfc21-c684-406a-9013-b6774bfb4f6a-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:21:44.320135 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:44.387424 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:21:44.393331 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] deleting rbd volume volume-f4acfc21-c684-406a-9013-b6774bfb4f6a {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:21:44.398178 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=64,cinder:UPDATE=24,cinder:INSERT=10,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:44.547481 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:21:44.579306 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] deleting parent snapshot volume-f4acfc21-c684-406a-9013-b6774bfb4f6a.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:21:45.152819 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:21:45.172440 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Flow 'volume_create_manager' (90f645b3-91a0-4ba0-bc6f-81bba7b884f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:45.179258 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e6b8db90-7c39-4d6b-bfa9-feb1ada4249d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.197569 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e6b8db90-7c39-4d6b-bfa9-feb1ada4249d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:21:44Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_eede824b-1fe2-4998-8fdf-84a59ace0173',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8f49ce01-740d-4958-95e8-e334234cf31f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='90cb48440cb446ea9abfc00ed0f53779',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:21:45Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:21:45Z,use_quota=True,user_id='55dadd491ec1441aac4171eff46bd679',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.202699 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8f593c08-e231-4b21-a520-7567b801916d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.206610 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8f593c08-e231-4b21-a520-7567b801916d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.211934 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fd6c9d2d-6cf4-4789-ab23-a5be39819031) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.215795 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fd6c9d2d-6cf4-4789-ab23-a5be39819031) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8f49ce01-740d-4958-95e8-e334234cf31f', 'volume_name': 'volume-8f49ce01-740d-4958-95e8-e334234cf31f', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.220397 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (593ecc56-0699-4138-8b7f-fa69a4c776ce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.224767 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (593ecc56-0699-4138-8b7f-fa69a4c776ce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.229179 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ed69c18d-6147-431e-b485-d24d7b8638d3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.230198 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Volume 8f49ce01-740d-4958-95e8-e334234cf31f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8f49ce01-740d-4958-95e8-e334234cf31f', 'volume_size': 1} Oct 01 21:21:45.230624 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] creating volume 'volume-8f49ce01-740d-4958-95e8-e334234cf31f' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:21:45.231358 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:45.252568 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:21:45.272399 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Flow 'volume_create_manager' (4e4203e3-3797-4162-9bc9-2a9847efb254) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:45.279109 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (33f42ee4-4e2f-47f5-bb75-7bb18dc24314) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.285761 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:21:45.286739 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:21:45.290113 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ed69c18d-6147-431e-b485-d24d7b8638d3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8f49ce01-740d-4958-95e8-e334234cf31f', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.294733 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ba55a0c6-5073-4b53-ad82-d07a97220dfb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.314310 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Volume volume-8f49ce01-740d-4958-95e8-e334234cf31f (8f49ce01-740d-4958-95e8-e334234cf31f): created successfully Oct 01 21:21:45.317925 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ba55a0c6-5073-4b53-ad82-d07a97220dfb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.323093 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Flow 'volume_create_manager' (90f645b3-91a0-4ba0-bc6f-81bba7b884f2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:45.330178 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (33f42ee4-4e2f-47f5-bb75-7bb18dc24314) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:21:45Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_26db3943-326a-45bf-8c90-6703318597d3',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='cca952fbe7c84efda0ecaf0dfb56a658',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:21:45Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:21:45Z,use_quota=True,user_id='733cf56b323f4844b6578ea1f9a46cfb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.335724 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9aa2eab6-d403-4888-be31-3b3557f0500b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.339613 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9aa2eab6-d403-4888-be31-3b3557f0500b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.344463 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c6cbe7de-dad4-4999-935c-b1fb952a65a9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.349093 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c6cbe7de-dad4-4999-935c-b1fb952a65a9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8', 'volume_name': 'volume-6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.353696 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7321441d-f4cf-4c4d-8f77-eebd564d498b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.357376 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7321441d-f4cf-4c4d-8f77-eebd564d498b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.362012 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8975ac11-7d1a-4080-bc16-5da29a31684e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.363058 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Volume 6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8', 'volume_size': 1} Oct 01 21:21:45.363446 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] creating volume 'volume-6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:21:45.364520 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:45.395050 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Created volume successfully. Oct 01 21:21:45.401298 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] Created reservations ['7b8ab8b5-277e-46cb-90dd-96f0dc4200e0', 'd88d2e2d-d50d-4cfb-88ef-a6eece9444e6', 'd47775e6-73b4-4ce2-8686-c1e77e950588', '27a992d5-b9a7-445e-95b6-cdcbf1a09c05'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:21:45.420416 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:21:45.420721 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:21:45.424337 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8975ac11-7d1a-4080-bc16-5da29a31684e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.428709 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (50613475-7ba6-4187-899e-6cbc9c7beb5d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:45.440712 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Volume volume-6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8 (6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8): created successfully Oct 01 21:21:45.444228 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (50613475-7ba6-4187-899e-6cbc9c7beb5d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:45.449408 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Flow 'volume_create_manager' (4e4203e3-3797-4162-9bc9-2a9847efb254) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:45.463525 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Created volume successfully. Oct 01 21:21:45.481340 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:45.505885 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:45.529747 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:21:45.533825 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] Deleted volume successfully. Oct 01 21:21:45.546349 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5d553b4e-bd27-424b-858d-f7f7b2184fe0 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-f4acfc21-c684-406a-9013-b6774bfb4f6a-delete_volume" released by "delete_volume" :: held 1.261s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:21:45.682351 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:21:46.139288 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:21:46.189177 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:21:46.211491 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (2089fcde-7d73-44d8-b21a-0de8bc30f325) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:46.218144 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3ab5d7da-ac7c-46b3-8d1b-f3aba8375e49) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:46.240396 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3ab5d7da-ac7c-46b3-8d1b-f3aba8375e49) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:21:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-564789299',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=af3590a1-ce01-4f8a-a2f8-cac49fea20f7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='df0e56b020444dd28af868590b1bccf4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:21:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:21:46Z,use_quota=True,user_id='eb0bf5fce885439eaa16a0871c2992ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:46.246413 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8f81619c-16d0-404d-a563-de7a8a36d279) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:46.250628 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8f81619c-16d0-404d-a563-de7a8a36d279) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:46.255906 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (844294a8-c4ef-40c8-b858-81269e3b8fd3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:46.345669 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (844294a8-c4ef-40c8-b858-81269e3b8fd3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'af3590a1-ce01-4f8a-a2f8-cac49fea20f7', 'volume_name': 'volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:46.351584 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (13c24339-4aba-4c55-a9b6-fd8775bef240) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:46.355492 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (13c24339-4aba-4c55-a9b6-fd8775bef240) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:46.360311 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (790eb338-0923-405e-bfc9-a52ae7efb621) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:46.361487 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Volume af3590a1-ce01-4f8a-a2f8-cac49fea20f7: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:21:46.362132 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Cloning af3590a1-ce01-4f8a-a2f8-cac49fea20f7 from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:21:46.362999 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:21:46.373657 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:21:46.410484 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 21, 46, 404566)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:21:46.411904 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:21:46.412491 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:21:46.416792 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.043s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:21:46.417889 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:21:46.424061 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 21, 46, 421755)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:21:46.424544 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:21:46.425553 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:21:46.438082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:46.509874 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] creating snapshot='volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:21:47.139311 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7.clone_snap' to 'volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:21:47.142182 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.460s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:21:47.142849 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.004s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:21:47.143565 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-8f49ce01-740d-4958-95e8-e334234cf31f', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '8f49ce01-740d-4958-95e8-e334234cf31f', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:21:47.146530 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:21:47.164736 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=70,cinder:DELETE=3,cinder:INSERT=4,cinder:UPDATE=23 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:47.170702 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Initialize volume connection completed successfully. Oct 01 21:21:47.172302 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Initialize volume connection completed successfully. Oct 01 21:21:47.306422 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:21:47.307394 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:21:47.307778 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:21:47.337604 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:21:47.338424 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Marking volume af3590a1-ce01-4f8a-a2f8-cac49fea20f7 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:21:47.350738 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to af3590a1-ce01-4f8a-a2f8-cac49fea20f7. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:21:47.351384 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Creating volume glance metadata for volume af3590a1-ce01-4f8a-a2f8-cac49fea20f7 backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:21:47.431228 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (790eb338-0923-405e-bfc9-a52ae7efb621) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:47.435810 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c4ee9348-2be5-4e49-a5e3-3966358d12a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:47.445842 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Volume volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7 (af3590a1-ce01-4f8a-a2f8-cac49fea20f7): created successfully Oct 01 21:21:47.449590 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c4ee9348-2be5-4e49-a5e3-3966358d12a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:47.455399 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (2089fcde-7d73-44d8-b21a-0de8bc30f325) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:47.464547 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-cfba3124-25fb-436a-8ea6-2f26f73464fc tempest-TestVolumeBootPattern-1545759599 None] Created volume successfully. Oct 01 21:21:48.960413 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Terminate volume connection completed successfully. Oct 01 21:21:48.999463 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4afbb5b5-1b53-4abd-a5ad-c4d8b71ea7a0 tempest-VolumesBackupsAdminTest-426242016 None] Remove volume export completed successfully. Oct 01 21:21:50.680832 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0d7aa9ce-5181-47f3-801c-757987811c87 tempest-VolumesBackupsAdminTest-426242016 None] Acquiring lock "cinder-6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:21:50.698921 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0d7aa9ce-5181-47f3-801c-757987811c87 tempest-VolumesBackupsAdminTest-426242016 None] Lock "cinder-6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8-delete_volume" acquired by "delete_volume" :: waited 0.025s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:21:50.741952 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0d7aa9ce-5181-47f3-801c-757987811c87 tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:51.417027 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0d7aa9ce-5181-47f3-801c-757987811c87 tempest-VolumesBackupsAdminTest-426242016 None] Volume volume-6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:21:51.515274 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0d7aa9ce-5181-47f3-801c-757987811c87 tempest-VolumesBackupsAdminTest-426242016 None] deleting rbd volume volume-6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:21:52.174735 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-8140adb4-6e14-4e56-b0b1-c69641b9537c req-b47b5f06-61b7-40df-a251-caff0f58bb45 tempest-TestVolumeBootPattern-1545759599 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:21:52.330965 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-0d7aa9ce-5181-47f3-801c-757987811c87 tempest-VolumesBackupsAdminTest-426242016 None] Created reservations ['a2c8f24e-a31f-4520-b1fe-7bf0939c0204', '6e7e4889-7e68-47ea-9488-a79d67600cba', 'a0fa6821-d46f-4248-8847-ad09456d4b71', 'a63d1e4f-acf4-4a82-85ba-24ba90c124ed'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:21:52.437198 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0d7aa9ce-5181-47f3-801c-757987811c87 tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:52.501965 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0d7aa9ce-5181-47f3-801c-757987811c87 tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:52.598464 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-0d7aa9ce-5181-47f3-801c-757987811c87 tempest-VolumesBackupsAdminTest-426242016 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:21:52.624854 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0d7aa9ce-5181-47f3-801c-757987811c87 tempest-VolumesBackupsAdminTest-426242016 None] Deleted volume successfully. Oct 01 21:21:52.648097 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-0d7aa9ce-5181-47f3-801c-757987811c87 tempest-VolumesBackupsAdminTest-426242016 None] Lock "cinder-6bb8e7d0-e954-42d5-a61c-c1e8ac4431a8-delete_volume" released by "delete_volume" :: held 1.949s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:21:53.084515 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Terminate volume connection completed successfully. Oct 01 21:21:53.122967 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-885781d3-4d34-429d-89fc-94dc8f9bf3ca tempest-TestVolumeBackupRestore-262965167 None] Remove volume export completed successfully. Oct 01 21:21:53.282247 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-8140adb4-6e14-4e56-b0b1-c69641b9537c req-b47b5f06-61b7-40df-a251-caff0f58bb45 tempest-TestVolumeBootPattern-1545759599 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.107s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:21:53.282967 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-8140adb4-6e14-4e56-b0b1-c69641b9537c req-b47b5f06-61b7-40df-a251-caff0f58bb45 tempest-TestVolumeBootPattern-1545759599 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'af3590a1-ce01-4f8a-a2f8-cac49fea20f7', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:21:53.329979 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-8140adb4-6e14-4e56-b0b1-c69641b9537c req-b47b5f06-61b7-40df-a251-caff0f58bb45 tempest-TestVolumeBootPattern-1545759599 None] Connection info returned from driver {'name': 'volumes/volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'af3590a1-ce01-4f8a-a2f8-cac49fea20f7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '4570efd7-096c-45e9-a2cf-d065cae57707'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:21:53.471996 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-8140adb4-6e14-4e56-b0b1-c69641b9537c req-b47b5f06-61b7-40df-a251-caff0f58bb45 tempest-TestVolumeBootPattern-1545759599 None] attachment_update completed successfully. Oct 01 21:21:53.703136 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=62,cinder:UPDATE=24,cinder:INSERT=12,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:21:53.715973 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:21:53.738868 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Flow 'volume_create_manager' (7bd38b1f-94ea-4b2f-b1e7-9e2127ccd7fc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:53.746262 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f8e531e3-16e4-46e5-9467-5b14000b7acb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:53.765211 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f8e531e3-16e4-46e5-9467-5b14000b7acb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:21:53Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Volume-1104779802',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4eacbbff-579e-493d-b963-056d94f09220,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='cca952fbe7c84efda0ecaf0dfb56a658',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:21:54Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:21:54Z,use_quota=True,user_id='733cf56b323f4844b6578ea1f9a46cfb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:53.770909 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cb6dde30-1db7-4ba0-8acf-75681d4505a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:53.775289 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cb6dde30-1db7-4ba0-8acf-75681d4505a3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:53.780237 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7b5fb4bf-5e3b-4b49-8064-02dad545762b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:53.784469 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7b5fb4bf-5e3b-4b49-8064-02dad545762b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4eacbbff-579e-493d-b963-056d94f09220', 'volume_name': 'volume-4eacbbff-579e-493d-b963-056d94f09220', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:53.789970 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (34ca1c08-c6c7-4054-a354-4b3dce4fc6bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:53.794329 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (34ca1c08-c6c7-4054-a354-4b3dce4fc6bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:53.799050 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0eb2dbfe-f15b-4fe5-b529-caa8a9130256) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:53.800115 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Volume 4eacbbff-579e-493d-b963-056d94f09220: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4eacbbff-579e-493d-b963-056d94f09220', 'volume_size': 1} Oct 01 21:21:53.800517 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] creating volume 'volume-4eacbbff-579e-493d-b963-056d94f09220' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:21:53.801820 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:21:53.874669 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:21:53.875306 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:21:53.879494 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0eb2dbfe-f15b-4fe5-b529-caa8a9130256) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4eacbbff-579e-493d-b963-056d94f09220', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:53.884107 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (391f9597-170f-4a59-b321-bc1f106735dc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:21:53.892231 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Volume volume-4eacbbff-579e-493d-b963-056d94f09220 (4eacbbff-579e-493d-b963-056d94f09220): created successfully Oct 01 21:21:53.895839 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (391f9597-170f-4a59-b321-bc1f106735dc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:21:53.901094 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Flow 'volume_create_manager' (7bd38b1f-94ea-4b2f-b1e7-9e2127ccd7fc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:21:53.908595 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c33cbba5-c43e-4c1a-8d7c-95084f8e72df tempest-VolumesBackupsAdminTest-426242016 None] Created volume successfully. Oct 01 21:21:54.991827 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d66d5b32-99b6-447f-bf07-c179bd4e7069 tempest-VolumesBackupsAdminTest-426242016 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-10-01T21:21:54Z,data_timestamp=2022-10-01T21:21:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-177874326',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=0ca05bde-2342-4f44-857b-60474c1765d0,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='cca952fbe7c84efda0ecaf0dfb56a658',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='733cf56b323f4844b6578ea1f9a46cfb',volume_id=4eacbbff-579e-493d-b963-056d94f09220) Oct 01 21:21:54.992610 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.backup.rpcapi [None req-d66d5b32-99b6-447f-bf07-c179bd4e7069 tempest-VolumesBackupsAdminTest-426242016 None] continue_backup in rpcapi backup_id 0ca05bde-2342-4f44-857b-60474c1765d0 {{(pid=133391) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Oct 01 21:21:55.129258 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-d66d5b32-99b6-447f-bf07-c179bd4e7069 tempest-VolumesBackupsAdminTest-426242016 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:21:56.002952 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-d66d5b32-99b6-447f-bf07-c179bd4e7069 tempest-VolumesBackupsAdminTest-426242016 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.866s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:21:56.002952 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d66d5b32-99b6-447f-bf07-c179bd4e7069 tempest-VolumesBackupsAdminTest-426242016 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-4eacbbff-579e-493d-b963-056d94f09220', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '4eacbbff-579e-493d-b963-056d94f09220', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:21:56.037765 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d66d5b32-99b6-447f-bf07-c179bd4e7069 tempest-VolumesBackupsAdminTest-426242016 None] Initialize volume connection completed successfully. Oct 01 21:21:58.047476 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-80e63e61-94d3-4f08-89af-1e74702bac0c req-3aa5aae3-bc5c-4f53-ae7a-6575ec3dcfe5 tempest-TestVolumeBackupRestore-262965167 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:21:58.222645 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d66d5b32-99b6-447f-bf07-c179bd4e7069 tempest-VolumesBackupsAdminTest-426242016 None] Terminate volume connection completed successfully. Oct 01 21:21:58.262472 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d66d5b32-99b6-447f-bf07-c179bd4e7069 tempest-VolumesBackupsAdminTest-426242016 None] Remove volume export completed successfully. Oct 01 21:21:58.907206 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-80e63e61-94d3-4f08-89af-1e74702bac0c req-3aa5aae3-bc5c-4f53-ae7a-6575ec3dcfe5 tempest-TestVolumeBackupRestore-262965167 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.860s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:21:58.909808 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-80e63e61-94d3-4f08-89af-1e74702bac0c req-3aa5aae3-bc5c-4f53-ae7a-6575ec3dcfe5 tempest-TestVolumeBackupRestore-262965167 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-8f49ce01-740d-4958-95e8-e334234cf31f', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '8f49ce01-740d-4958-95e8-e334234cf31f', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:21:58.943309 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-80e63e61-94d3-4f08-89af-1e74702bac0c req-3aa5aae3-bc5c-4f53-ae7a-6575ec3dcfe5 tempest-TestVolumeBackupRestore-262965167 None] Connection info returned from driver {'name': 'volumes/volume-8f49ce01-740d-4958-95e8-e334234cf31f', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '8f49ce01-740d-4958-95e8-e334234cf31f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '17f98f36-1a7c-472f-a5de-5c25e281de00'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:21:59.025236 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-80e63e61-94d3-4f08-89af-1e74702bac0c req-3aa5aae3-bc5c-4f53-ae7a-6575ec3dcfe5 tempest-TestVolumeBackupRestore-262965167 None] attachment_update completed successfully. Oct 01 21:22:00.441255 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-64fb6071-169c-42d7-a510-a894cf463233 tempest-VolumesBackupsAdminTest-426242016 None] Acquiring lock "cinder-4eacbbff-579e-493d-b963-056d94f09220-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:22:00.450611 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-64fb6071-169c-42d7-a510-a894cf463233 tempest-VolumesBackupsAdminTest-426242016 None] Lock "cinder-4eacbbff-579e-493d-b963-056d94f09220-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:22:00.490935 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-64fb6071-169c-42d7-a510-a894cf463233 tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:01.034367 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-64fb6071-169c-42d7-a510-a894cf463233 tempest-VolumesBackupsAdminTest-426242016 None] Volume volume-4eacbbff-579e-493d-b963-056d94f09220 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:22:01.051916 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-64fb6071-169c-42d7-a510-a894cf463233 tempest-VolumesBackupsAdminTest-426242016 None] deleting rbd volume volume-4eacbbff-579e-493d-b963-056d94f09220 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:22:01.300478 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-64fb6071-169c-42d7-a510-a894cf463233 tempest-VolumesBackupsAdminTest-426242016 None] Created reservations ['4870270a-5ab3-4ae6-8e02-058b0287a36a', '1ef84806-f95f-45e8-8176-563559c8ea37', '3072eb02-7f9d-4240-816e-f61f8531b4c6', '5fa2eae4-9d3c-4af7-a0e8-a352554c8a68'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:22:01.308930 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=71,cinder:UPDATE=19,cinder:INSERT=9,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:01.356121 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-64fb6071-169c-42d7-a510-a894cf463233 tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:01.390830 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-64fb6071-169c-42d7-a510-a894cf463233 tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:01.434395 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-64fb6071-169c-42d7-a510-a894cf463233 tempest-VolumesBackupsAdminTest-426242016 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:22:01.439223 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-64fb6071-169c-42d7-a510-a894cf463233 tempest-VolumesBackupsAdminTest-426242016 None] Deleted volume successfully. Oct 01 21:22:01.461685 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-64fb6071-169c-42d7-a510-a894cf463233 tempest-VolumesBackupsAdminTest-426242016 None] Lock "cinder-4eacbbff-579e-493d-b963-056d94f09220-delete_volume" released by "delete_volume" :: held 1.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:22:01.577802 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f386c357-f1ea-491d-a270-3a8b26e4b0ca tempest-VolumesBackupsAdminTest-426242016 None] Acquiring lock "cinder-7879d8cf-be58-4353-baa6-ec93c3a8634a-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:22:01.588024 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f386c357-f1ea-491d-a270-3a8b26e4b0ca tempest-VolumesBackupsAdminTest-426242016 None] Lock "cinder-7879d8cf-be58-4353-baa6-ec93c3a8634a-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:22:01.624683 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f386c357-f1ea-491d-a270-3a8b26e4b0ca tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:02.039474 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f386c357-f1ea-491d-a270-3a8b26e4b0ca tempest-VolumesBackupsAdminTest-426242016 None] Volume volume-7879d8cf-be58-4353-baa6-ec93c3a8634a is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:22:02.054330 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f386c357-f1ea-491d-a270-3a8b26e4b0ca tempest-VolumesBackupsAdminTest-426242016 None] deleting rbd volume volume-7879d8cf-be58-4353-baa6-ec93c3a8634a {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:22:02.250186 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-f386c357-f1ea-491d-a270-3a8b26e4b0ca tempest-VolumesBackupsAdminTest-426242016 None] Created reservations ['43f39d2b-01cc-4717-af29-58bfd298d917', 'a8fbd927-afd2-41cc-8181-e619dd109383', '0cb6bb73-be13-4dca-a208-fcbe295e67f1', '0a866597-f354-40f1-9203-e12daecfb90d'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:22:02.292019 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f386c357-f1ea-491d-a270-3a8b26e4b0ca tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:02.316836 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f386c357-f1ea-491d-a270-3a8b26e4b0ca tempest-VolumesBackupsAdminTest-426242016 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:02.360505 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-f386c357-f1ea-491d-a270-3a8b26e4b0ca tempest-VolumesBackupsAdminTest-426242016 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:22:02.365996 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f386c357-f1ea-491d-a270-3a8b26e4b0ca tempest-VolumesBackupsAdminTest-426242016 None] Deleted volume successfully. Oct 01 21:22:02.378703 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f386c357-f1ea-491d-a270-3a8b26e4b0ca tempest-VolumesBackupsAdminTest-426242016 None] Lock "cinder-7879d8cf-be58-4353-baa6-ec93c3a8634a-delete_volume" released by "delete_volume" :: held 0.791s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:22:03.296141 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-b0c37e8e-dcc3-4057-8c45-a5a6b2fbb0ec req-26afbde8-3864-40e8-bf5b-017bee0b1457 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:04.218349 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-b0c37e8e-dcc3-4057-8c45-a5a6b2fbb0ec req-26afbde8-3864-40e8-bf5b-017bee0b1457 tempest-TestVolumeBootPattern-1545759599 None] Create snapshot completed successfully Oct 01 21:22:06.564146 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:22:06.585341 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Flow 'volume_create_manager' (92619c84-01ca-4cc0-91c5-f8f1927c8ecd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:06.593449 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3e6e191d-86ad-423c-b2c9-c6747bf0acaf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:06.610617 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3e6e191d-86ad-423c-b2c9-c6747bf0acaf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:22:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-494436145',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cc260b31-975b-4074-9861-d884a4bbf016,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='410af66d56c34146994cac962b73fbfd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:22:06Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:22:07Z,use_quota=True,user_id='22c02150477f4e98a9aa7fd0247def02',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:06.616192 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8859e2ea-3c65-4523-91cb-873e69b6c30b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:06.620540 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8859e2ea-3c65-4523-91cb-873e69b6c30b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:06.625379 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e5185569-cc3d-48c2-bc42-265a64cb80a6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:06.629819 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e5185569-cc3d-48c2-bc42-265a64cb80a6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'cc260b31-975b-4074-9861-d884a4bbf016', 'volume_name': 'volume-cc260b31-975b-4074-9861-d884a4bbf016', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:06.634782 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (567639c7-c99f-4963-9e3e-47809d552274) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:06.638515 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (567639c7-c99f-4963-9e3e-47809d552274) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:06.643100 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d0ea7b7f-22b8-40ca-88d9-482528089e7a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:06.644166 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Volume cc260b31-975b-4074-9861-d884a4bbf016: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-cc260b31-975b-4074-9861-d884a4bbf016', 'volume_size': 1} Oct 01 21:22:06.645168 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] creating volume 'volume-cc260b31-975b-4074-9861-d884a4bbf016' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:22:06.646120 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:06.717429 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:22:06.717803 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:22:06.721718 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d0ea7b7f-22b8-40ca-88d9-482528089e7a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cc260b31-975b-4074-9861-d884a4bbf016', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:06.726384 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (13a04427-3236-4170-b7cd-e99140c343a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:06.737158 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Volume volume-cc260b31-975b-4074-9861-d884a4bbf016 (cc260b31-975b-4074-9861-d884a4bbf016): created successfully Oct 01 21:22:06.741358 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (13a04427-3236-4170-b7cd-e99140c343a8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:06.747720 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Flow 'volume_create_manager' (92619c84-01ca-4cc0-91c5-f8f1927c8ecd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:06.758212 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0163b54d-d092-4c97-b12d-f154729c1a94 tempest-VolumesListAdminTestJSON-853391859 None] Created volume successfully. Oct 01 21:22:07.036064 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:22:07.064950 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (eaa44189-1d16-45ef-88b8-2bb32eb078d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:07.071953 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (428b3c05-2dfd-4169-a447-ec37494e2f97) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:07.089391 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (428b3c05-2dfd-4169-a447-ec37494e2f97) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:22:07Z,deleted=False,deleted_at=None,display_description='',display_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=0ee81fee-f2c0-4d11-8269-891242b96cb3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='df0e56b020444dd28af868590b1bccf4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:22:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=67c87bce-6167-4219-9eee-20a0aeed139b,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:22:07Z,use_quota=True,user_id='eb0bf5fce885439eaa16a0871c2992ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:07.094542 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (53b00d08-bf27-4338-91ae-b46fa3a5064f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:07.098546 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (53b00d08-bf27-4338-91ae-b46fa3a5064f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:07.103747 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cf029b8b-a0ff-4e8a-87c6-2b6ddccfd206) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:07.107878 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cf029b8b-a0ff-4e8a-87c6-2b6ddccfd206) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '0ee81fee-f2c0-4d11-8269-891242b96cb3', 'volume_name': 'volume-0ee81fee-f2c0-4d11-8269-891242b96cb3', 'volume_size': 1, 'snapshot_id': '67c87bce-6167-4219-9eee-20a0aeed139b'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:07.114072 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (426dc9aa-405c-401f-b6a7-08440f26465d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:07.118131 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (426dc9aa-405c-401f-b6a7-08440f26465d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:07.122642 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3413f20f-f9a1-442a-ab3a-15b278801c21) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:07.123695 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Volume 0ee81fee-f2c0-4d11-8269-891242b96cb3: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-0ee81fee-f2c0-4d11-8269-891242b96cb3', 'volume_size': 1, 'snapshot_id': '67c87bce-6167-4219-9eee-20a0aeed139b'} Oct 01 21:22:07.150554 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] cloning volumes/volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7@snapshot-67c87bce-6167-4219-9eee-20a0aeed139b to volume-0ee81fee-f2c0-4d11-8269-891242b96cb3 {{(pid=133391) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Oct 01 21:22:07.150989 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:07.228212 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:07.380844 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:22:07.380844 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:22:07.402530 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Marking volume 0ee81fee-f2c0-4d11-8269-891242b96cb3 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:22:07.407705 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Copying metadata from snapshot 67c87bce-6167-4219-9eee-20a0aeed139b to 0ee81fee-f2c0-4d11-8269-891242b96cb3. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:431}} Oct 01 21:22:07.424283 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:DELETE=2,cinder:SELECT=62,cinder:UPDATE=24,cinder:INSERT=12 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:07.445949 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3413f20f-f9a1-442a-ab3a-15b278801c21) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-0ee81fee-f2c0-4d11-8269-891242b96cb3', 'volume_size': 1, 'snapshot_id': '67c87bce-6167-4219-9eee-20a0aeed139b'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:07.452594 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1a78ef54-90e0-4373-ab03-a0de457bfd25) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:07.467157 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Volume volume-0ee81fee-f2c0-4d11-8269-891242b96cb3 (0ee81fee-f2c0-4d11-8269-891242b96cb3): created successfully Oct 01 21:22:07.471948 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1a78ef54-90e0-4373-ab03-a0de457bfd25) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:07.478394 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (eaa44189-1d16-45ef-88b8-2bb32eb078d1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:07.497945 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-866082a6-3479-468f-b38e-51dbce64b65e tempest-TestVolumeBootPattern-1545759599 None] Created volume successfully. Oct 01 21:22:08.090395 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:22:08.114164 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Flow 'volume_create_manager' (20910d40-8e02-4883-9c57-b9fbd55e9647) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:08.121001 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c07b8146-a480-4764-9898-1daa8cdecc82) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:08.140259 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c07b8146-a480-4764-9898-1daa8cdecc82) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:22:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-1790405421',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=df719679-0a3f-485b-a56e-e845607d725d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='410af66d56c34146994cac962b73fbfd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:22:08Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:22:08Z,use_quota=True,user_id='22c02150477f4e98a9aa7fd0247def02',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:08.146920 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5d543701-08f1-497c-ac0b-4bca82fd69d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:08.151465 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5d543701-08f1-497c-ac0b-4bca82fd69d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:08.157287 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (420eb541-0fae-4a31-8412-6806c1117047) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:08.161931 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (420eb541-0fae-4a31-8412-6806c1117047) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'df719679-0a3f-485b-a56e-e845607d725d', 'volume_name': 'volume-df719679-0a3f-485b-a56e-e845607d725d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:08.166015 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e28becd4-19d0-4f5d-aefa-e21af5f8bbd6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:08.171616 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e28becd4-19d0-4f5d-aefa-e21af5f8bbd6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:08.176198 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e6c6ed92-6bea-43bb-9862-ce229fa13909) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:08.177273 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Volume df719679-0a3f-485b-a56e-e845607d725d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-df719679-0a3f-485b-a56e-e845607d725d', 'volume_size': 1} Oct 01 21:22:08.177668 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] creating volume 'volume-df719679-0a3f-485b-a56e-e845607d725d' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:22:08.178323 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:08.254224 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:22:08.254704 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:22:08.259108 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e6c6ed92-6bea-43bb-9862-ce229fa13909) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-df719679-0a3f-485b-a56e-e845607d725d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:08.265369 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b3e034da-2fba-456d-883d-177e4aab3acb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:08.277861 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Volume volume-df719679-0a3f-485b-a56e-e845607d725d (df719679-0a3f-485b-a56e-e845607d725d): created successfully Oct 01 21:22:08.285673 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b3e034da-2fba-456d-883d-177e4aab3acb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:08.288905 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Flow 'volume_create_manager' (20910d40-8e02-4883-9c57-b9fbd55e9647) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:08.307412 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5e382166-a100-4036-be06-3fec422f71f5 tempest-VolumesListAdminTestJSON-853391859 None] Created volume successfully. Oct 01 21:22:09.479887 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:22:09.507195 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Flow 'volume_create_manager' (f32f9bcb-b8a2-4b2e-89d3-d9814e3ad8ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:09.514811 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e2a3d11b-7ebb-46b2-89ac-a655cd2a2a2c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:09.531864 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e2a3d11b-7ebb-46b2-89ac-a655cd2a2a2c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:22:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-1466503192',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1bb1c7d6-3be9-40a0-a42b-393b75fa6c33,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='410af66d56c34146994cac962b73fbfd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:22:09Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:22:09Z,use_quota=True,user_id='22c02150477f4e98a9aa7fd0247def02',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:09.537605 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f46fb0b3-c620-4c6f-88d2-440996198a83) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:09.541790 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f46fb0b3-c620-4c6f-88d2-440996198a83) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:09.547401 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8cb8b599-f212-4262-8e44-da9a7e801586) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:09.552513 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8cb8b599-f212-4262-8e44-da9a7e801586) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1bb1c7d6-3be9-40a0-a42b-393b75fa6c33', 'volume_name': 'volume-1bb1c7d6-3be9-40a0-a42b-393b75fa6c33', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:09.557606 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8aa30e5e-8806-4e81-a05a-6d2f504c8eb1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:09.561587 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8aa30e5e-8806-4e81-a05a-6d2f504c8eb1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:09.566834 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a1e48ebb-2116-4964-b345-9a4a2f720bb0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:09.567985 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Volume 1bb1c7d6-3be9-40a0-a42b-393b75fa6c33: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1bb1c7d6-3be9-40a0-a42b-393b75fa6c33', 'volume_size': 1} Oct 01 21:22:09.568462 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] creating volume 'volume-1bb1c7d6-3be9-40a0-a42b-393b75fa6c33' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:22:09.569190 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:09.640441 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:22:09.641550 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:22:09.646522 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a1e48ebb-2116-4964-b345-9a4a2f720bb0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1bb1c7d6-3be9-40a0-a42b-393b75fa6c33', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:09.653457 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (395dcd87-fd06-4c5b-bd41-fb0c5908e78a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:09.664784 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Volume volume-1bb1c7d6-3be9-40a0-a42b-393b75fa6c33 (1bb1c7d6-3be9-40a0-a42b-393b75fa6c33): created successfully Oct 01 21:22:09.668955 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (395dcd87-fd06-4c5b-bd41-fb0c5908e78a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:09.675118 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Flow 'volume_create_manager' (f32f9bcb-b8a2-4b2e-89d3-d9814e3ad8ee) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:09.684670 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d9bb0bdf-2b49-486e-9bf2-f59124b62006 tempest-VolumesListAdminTestJSON-853391859 None] Created volume successfully. Oct 01 21:22:10.310418 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-788f1e90-bb72-4317-8844-11a4ff3de057 tempest-TestVolumeBootPattern-1545759599 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:22:11.254876 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:22:11.280853 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Flow 'volume_create_manager' (1026c6ed-0a47-429a-9fbc-4a29379aba7e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:11.287968 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4c2db3df-a925-4e9c-b4a9-8559d050b4d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:11.293236 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-788f1e90-bb72-4317-8844-11a4ff3de057 tempest-TestVolumeBootPattern-1545759599 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.983s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:22:11.294934 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-788f1e90-bb72-4317-8844-11a4ff3de057 tempest-TestVolumeBootPattern-1545759599 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-0ee81fee-f2c0-4d11-8269-891242b96cb3', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '0ee81fee-f2c0-4d11-8269-891242b96cb3', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:22:11.313507 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4c2db3df-a925-4e9c-b4a9-8559d050b4d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:22:11Z,deleted=False,deleted_at=None,display_description=None,display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a87b6287-3913-46a5-bce2-b179aa528c71,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='12e7b51181e04f3eb5dcc48e45dabde5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:22:11Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:22:11Z,use_quota=True,user_id='51642c736773414886ad03bd940c8778',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:11.319512 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0d06db61-f0d0-4f49-bae9-864867be4ed3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:11.323617 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0d06db61-f0d0-4f49-bae9-864867be4ed3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:11.328725 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (14fe3fb2-7850-438c-a6da-2e770ef0ffaa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:11.333454 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (14fe3fb2-7850-438c-a6da-2e770ef0ffaa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a87b6287-3913-46a5-bce2-b179aa528c71', 'volume_name': 'volume-a87b6287-3913-46a5-bce2-b179aa528c71', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:11.338530 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bed20e17-18d9-495e-b547-0294d9b8388a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:11.342614 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bed20e17-18d9-495e-b547-0294d9b8388a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:11.347407 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9baf1732-49bc-4a7e-a665-7c2dc3be3412) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:11.348558 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Volume a87b6287-3913-46a5-bce2-b179aa528c71: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a87b6287-3913-46a5-bce2-b179aa528c71', 'volume_size': 1} Oct 01 21:22:11.349042 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] creating volume 'volume-a87b6287-3913-46a5-bce2-b179aa528c71' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:22:11.349878 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:11.408920 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-788f1e90-bb72-4317-8844-11a4ff3de057 tempest-TestVolumeBootPattern-1545759599 None] Connection info returned from driver {'name': 'volumes/volume-0ee81fee-f2c0-4d11-8269-891242b96cb3', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '0ee81fee-f2c0-4d11-8269-891242b96cb3', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '325fcea8-74e0-4595-888a-6945d2e0c288'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:22:11.447818 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:22:11.448420 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:22:11.452498 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9baf1732-49bc-4a7e-a665-7c2dc3be3412) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a87b6287-3913-46a5-bce2-b179aa528c71', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:11.458749 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a25d4fa2-9d1b-4ee6-bf33-0790933185d5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:11.486509 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Volume volume-a87b6287-3913-46a5-bce2-b179aa528c71 (a87b6287-3913-46a5-bce2-b179aa528c71): created successfully Oct 01 21:22:11.490807 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a25d4fa2-9d1b-4ee6-bf33-0790933185d5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:11.496873 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Flow 'volume_create_manager' (1026c6ed-0a47-429a-9fbc-4a29379aba7e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:11.507844 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-2c226c54-da8c-4300-b8af-9cad124e4523 tempest-VolumesListAdminTestJSON-856594610 None] Created volume successfully. Oct 01 21:22:11.524611 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=58,cinder:UPDATE=21,cinder:INSERT=18,cinder:DELETE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:11.527099 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-fbc2b699-3aef-49fa-9e4e-b612ae0fbc40 req-788f1e90-bb72-4317-8844-11a4ff3de057 tempest-TestVolumeBootPattern-1545759599 None] attachment_update completed successfully. Oct 01 21:22:12.356031 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] Acquiring lock "cinder-a87b6287-3913-46a5-bce2-b179aa528c71-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:22:12.367115 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] Lock "cinder-a87b6287-3913-46a5-bce2-b179aa528c71-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:22:12.399146 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:12.467590 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:22:12.468049 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] Volume volume-a87b6287-3913-46a5-bce2-b179aa528c71 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:22:12.475113 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] deleting rbd volume volume-a87b6287-3913-46a5-bce2-b179aa528c71 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:22:12.490040 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] Acquiring lock "cinder-1bb1c7d6-3be9-40a0-a42b-393b75fa6c33-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:22:12.502129 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] Lock "cinder-1bb1c7d6-3be9-40a0-a42b-393b75fa6c33-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:22:12.558854 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:12.637918 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:22:12.639139 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] Volume volume-1bb1c7d6-3be9-40a0-a42b-393b75fa6c33 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:22:12.645640 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] deleting rbd volume volume-1bb1c7d6-3be9-40a0-a42b-393b75fa6c33 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:22:12.948367 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] Created reservations ['887f01b4-cb6b-41f0-bbdf-f88052648f65', 'cf95a41f-9ba8-456d-ba28-1d1b9133eea5', 'ee009db2-d62a-4674-aa8c-a7ec2b6b1c65', '4406632c-f8fd-4c00-b977-d8e9fc8e5b8c'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:22:13.048836 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:13.084196 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:13.131480 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:22:13.137595 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] Deleted volume successfully. Oct 01 21:22:13.168028 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a225863c-a405-438c-aa0f-c6f538f7df2c tempest-VolumesListAdminTestJSON-856594610 None] Lock "cinder-a87b6287-3913-46a5-bce2-b179aa528c71-delete_volume" released by "delete_volume" :: held 0.801s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:22:13.202477 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] Created reservations ['8d239568-df35-4b62-bd00-dde348752ff0', '61294e8d-4864-4b49-8c8e-661cdfa77853', '47d705ee-f0e1-438c-b9e9-ba9a4e60a827', '352e99ae-2f22-4a2b-85a0-14799089bf6c'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:22:13.275709 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:13.319912 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:13.358430 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:22:13.364287 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] Deleted volume successfully. Oct 01 21:22:13.377720 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5a7238d8-9a87-4d2c-9a59-4297fe43d7f6 tempest-VolumesListAdminTestJSON-853391859 None] Lock "cinder-1bb1c7d6-3be9-40a0-a42b-393b75fa6c33-delete_volume" released by "delete_volume" :: held 0.876s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:22:13.585953 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] Acquiring lock "cinder-df719679-0a3f-485b-a56e-e845607d725d-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:22:13.608325 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] Lock "cinder-df719679-0a3f-485b-a56e-e845607d725d-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:22:13.667330 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:13.723569 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:22:13.724027 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] Volume volume-df719679-0a3f-485b-a56e-e845607d725d is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:22:13.729967 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] deleting rbd volume volume-df719679-0a3f-485b-a56e-e845607d725d {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:22:13.916091 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=65,cinder:DELETE=5,cinder:UPDATE=27,cinder:INSERT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:13.942668 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] Created reservations ['1165d6f7-2a9a-4f9c-bf60-a9c3260b9d69', 'bfb5d945-a8eb-4303-8d27-6985f70db47c', 'b714b924-c712-4084-a77c-719d5c042293', '219c2b78-8a77-4a2e-97f4-82976b608286'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:22:13.989330 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:14.013558 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:14.048129 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:22:14.053639 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] Deleted volume successfully. Oct 01 21:22:14.068148 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9230cf05-1f02-4c46-a004-7fae01cbd619 tempest-VolumesListAdminTestJSON-853391859 None] Lock "cinder-df719679-0a3f-485b-a56e-e845607d725d-delete_volume" released by "delete_volume" :: held 0.460s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:22:14.733712 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] Acquiring lock "cinder-cc260b31-975b-4074-9861-d884a4bbf016-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:22:14.748818 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] Lock "cinder-cc260b31-975b-4074-9861-d884a4bbf016-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:22:14.783351 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:14.829115 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:22:14.829684 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] Volume volume-cc260b31-975b-4074-9861-d884a4bbf016 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:22:14.836220 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] deleting rbd volume volume-cc260b31-975b-4074-9861-d884a4bbf016 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:22:15.108952 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] Created reservations ['200de902-b0cb-4bb8-8afa-4928088e60b9', '70ddd2fc-c807-4302-adbc-329257acd0c1', '41e6be57-0da6-4903-9588-54d81206cbbb', 'c71b79c7-959d-4a69-b4d5-a90caa7111bd'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:22:15.155530 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:15.184843 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:15.213122 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:22:15.217652 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] Deleted volume successfully. Oct 01 21:22:15.230968 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1020316e-474e-406d-84b4-3bbe4227b6a6 tempest-VolumesListAdminTestJSON-853391859 None] Lock "cinder-cc260b31-975b-4074-9861-d884a4bbf016-delete_volume" released by "delete_volume" :: held 0.482s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:22:18.227174 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:22:18.246038 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Flow 'volume_create_manager' (86a9a445-437b-4b1c-8be4-ee977d75bcf1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:18.252234 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ced279d7-9e54-47c2-8363-dd90882bb79c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:18.267626 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ced279d7-9e54-47c2-8363-dd90882bb79c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:22:18Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1621032150',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2c387064-4e94-46fa-8a41-d63dfab6a47f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0bb47fed88494698a3be2a0b34761464',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:22:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:22:18Z,use_quota=True,user_id='2f6a9ae0c03d4328be8a4665cc503005',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:18.273272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d88b4811-1a3d-4261-a435-f3bef274c889) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:18.277263 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d88b4811-1a3d-4261-a435-f3bef274c889) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:18.282205 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b0c972eb-be17-4f9f-812f-d9409404ddff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:18.287035 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b0c972eb-be17-4f9f-812f-d9409404ddff) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2c387064-4e94-46fa-8a41-d63dfab6a47f', 'volume_name': 'volume-2c387064-4e94-46fa-8a41-d63dfab6a47f', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:18.292913 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a72a143-9bbc-41d7-b6a1-22c81e8eef67) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:18.297018 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a72a143-9bbc-41d7-b6a1-22c81e8eef67) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:18.301608 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dfbbc39a-b4e9-42b2-aa5b-8fb5507ee99b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:18.302691 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Volume 2c387064-4e94-46fa-8a41-d63dfab6a47f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2c387064-4e94-46fa-8a41-d63dfab6a47f', 'volume_size': 1} Oct 01 21:22:18.303085 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] creating volume 'volume-2c387064-4e94-46fa-8a41-d63dfab6a47f' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:22:18.303859 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:18.370269 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:22:18.370740 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:22:18.375322 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dfbbc39a-b4e9-42b2-aa5b-8fb5507ee99b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2c387064-4e94-46fa-8a41-d63dfab6a47f', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:18.380396 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (109d26fd-28e9-48a5-8530-54999d9edab7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:18.390376 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Volume volume-2c387064-4e94-46fa-8a41-d63dfab6a47f (2c387064-4e94-46fa-8a41-d63dfab6a47f): created successfully Oct 01 21:22:18.394466 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (109d26fd-28e9-48a5-8530-54999d9edab7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:18.399917 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Flow 'volume_create_manager' (86a9a445-437b-4b1c-8be4-ee977d75bcf1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:18.407817 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-df8a017b-31d4-4690-b8b2-99f4d1736914 tempest-VolumesBackupsTest-1744874988 None] Created volume successfully. Oct 01 21:22:23.813693 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:22:23.814416 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:23.842435 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:23.869983 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:22:24.405627 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=56,cinder:INSERT=11,cinder:UPDATE=31,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:25.538919 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-0abf805a-62a7-4533-acaa-8f0a3ee8526b tempest-TestVolumeBootPattern-1545759599 None] Terminate volume connection completed successfully. Oct 01 21:22:25.539644 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-0abf805a-62a7-4533-acaa-8f0a3ee8526b tempest-TestVolumeBootPattern-1545759599 None] Deleting attachment 325fcea8-74e0-4595-888a-6945d2e0c288. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:22:25.701393 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-8414deae-0d81-4132-b839-662bcd21a2c9 tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-0ee81fee-f2c0-4d11-8269-891242b96cb3-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:22:25.713497 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-8414deae-0d81-4132-b839-662bcd21a2c9 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-0ee81fee-f2c0-4d11-8269-891242b96cb3-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:22:25.770892 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-8414deae-0d81-4132-b839-662bcd21a2c9 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:25.903639 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-8414deae-0d81-4132-b839-662bcd21a2c9 tempest-TestVolumeBootPattern-1545759599 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:22:25.915342 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-8414deae-0d81-4132-b839-662bcd21a2c9 tempest-TestVolumeBootPattern-1545759599 None] deleting rbd volume volume-0ee81fee-f2c0-4d11-8269-891242b96cb3 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:22:26.223619 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-8414deae-0d81-4132-b839-662bcd21a2c9 tempest-TestVolumeBootPattern-1545759599 None] Created reservations ['b6787a66-5256-4203-8118-1d5fd672d9ec', '8c0b4336-f2bb-47f4-ae87-405e4ffbdc53', '0b7bb494-59f7-417c-8b5e-fcc15e206119', '792b51b3-2df7-43bc-9597-7c12832331d4'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:22:26.272644 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-8414deae-0d81-4132-b839-662bcd21a2c9 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:26.298636 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-8414deae-0d81-4132-b839-662bcd21a2c9 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:26.326067 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-8414deae-0d81-4132-b839-662bcd21a2c9 tempest-TestVolumeBootPattern-1545759599 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:22:26.330420 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-8414deae-0d81-4132-b839-662bcd21a2c9 tempest-TestVolumeBootPattern-1545759599 None] Deleted volume successfully. Oct 01 21:22:26.343368 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-1138b26c-ee4b-4901-bfa0-6a47ae03fc6a req-8414deae-0d81-4132-b839-662bcd21a2c9 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-0ee81fee-f2c0-4d11-8269-891242b96cb3-delete_volume" released by "delete_volume" :: held 0.630s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:22:27.792826 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-2d4c4616-fe2d-4929-86c5-2d8a3125b891 tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-67c87bce-6167-4219-9eee-20a0aeed139b-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:22:27.808957 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-2d4c4616-fe2d-4929-86c5-2d8a3125b891 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-67c87bce-6167-4219-9eee-20a0aeed139b-delete_snapshot" acquired by "delete_snapshot" :: waited 0.017s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:22:27.811144 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2d4c4616-fe2d-4929-86c5-2d8a3125b891 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:32.409179 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-ac3d02db-d918-43ea-bcc9-31de38813b5d req-e890ec53-c875-4f03-ada3-7ab43215e8f0 tempest-VolumesBackupsTest-1744874988 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:22:33.193718 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-ac3d02db-d918-43ea-bcc9-31de38813b5d req-e890ec53-c875-4f03-ada3-7ab43215e8f0 tempest-VolumesBackupsTest-1744874988 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.783s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:22:33.195258 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-ac3d02db-d918-43ea-bcc9-31de38813b5d req-e890ec53-c875-4f03-ada3-7ab43215e8f0 tempest-VolumesBackupsTest-1744874988 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2c387064-4e94-46fa-8a41-d63dfab6a47f', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '2c387064-4e94-46fa-8a41-d63dfab6a47f', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:22:33.222075 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-ac3d02db-d918-43ea-bcc9-31de38813b5d req-e890ec53-c875-4f03-ada3-7ab43215e8f0 tempest-VolumesBackupsTest-1744874988 None] Connection info returned from driver {'name': 'volumes/volume-2c387064-4e94-46fa-8a41-d63dfab6a47f', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '2c387064-4e94-46fa-8a41-d63dfab6a47f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '5b888ecf-6336-4f50-a768-cb524e95dd30'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:22:33.276995 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-ac3d02db-d918-43ea-bcc9-31de38813b5d req-e890ec53-c875-4f03-ada3-7ab43215e8f0 tempest-VolumesBackupsTest-1744874988 None] attachment_update completed successfully. Oct 01 21:22:35.722231 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8ea0f31b-ac5c-45ab-9057-05f62c688cbd tempest-VolumesBackupsTest-1744874988 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-10-01T21:22:35Z,data_timestamp=2022-10-01T21:22:35Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-1354899700',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=13afe5e6-5181-4f63-8774-fb73ee85d203,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='0bb47fed88494698a3be2a0b34761464',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='2f6a9ae0c03d4328be8a4665cc503005',volume_id=2c387064-4e94-46fa-8a41-d63dfab6a47f) Oct 01 21:22:35.723267 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.backup.rpcapi [None req-8ea0f31b-ac5c-45ab-9057-05f62c688cbd tempest-VolumesBackupsTest-1744874988 None] continue_backup in rpcapi backup_id 13afe5e6-5181-4f63-8774-fb73ee85d203 {{(pid=133391) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Oct 01 21:22:35.822279 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-8ea0f31b-ac5c-45ab-9057-05f62c688cbd tempest-VolumesBackupsTest-1744874988 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:22:36.685680 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-8ea0f31b-ac5c-45ab-9057-05f62c688cbd tempest-VolumesBackupsTest-1744874988 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.862s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:22:36.687021 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8ea0f31b-ac5c-45ab-9057-05f62c688cbd tempest-VolumesBackupsTest-1744874988 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2c387064-4e94-46fa-8a41-d63dfab6a47f', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '2c387064-4e94-46fa-8a41-d63dfab6a47f', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:22:36.705128 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8ea0f31b-ac5c-45ab-9057-05f62c688cbd tempest-VolumesBackupsTest-1744874988 None] Initialize volume connection completed successfully. Oct 01 21:22:38.579340 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8ea0f31b-ac5c-45ab-9057-05f62c688cbd tempest-VolumesBackupsTest-1744874988 None] Terminate volume connection completed successfully. Oct 01 21:22:38.598331 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8ea0f31b-ac5c-45ab-9057-05f62c688cbd tempest-VolumesBackupsTest-1744874988 None] Remove volume export completed successfully. Oct 01 21:22:40.906361 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-bbf814b4-9599-40c4-a033-3e80432efa56 req-9202edc5-b16b-4c75-9a06-7f80903300cb tempest-VolumesBackupsTest-1744874988 None] Terminate volume connection completed successfully. Oct 01 21:22:40.907202 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-bbf814b4-9599-40c4-a033-3e80432efa56 req-9202edc5-b16b-4c75-9a06-7f80903300cb tempest-VolumesBackupsTest-1744874988 None] Deleting attachment 5b888ecf-6336-4f50-a768-cb524e95dd30. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:22:44.409384 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=30,cinder:SELECT=50,cinder:DELETE=3,cinder:INSERT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:45.082805 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9d12dac5-c8d7-4cc0-801d-430781035787 tempest-VolumesBackupsTest-1744874988 None] Acquiring lock "cinder-2c387064-4e94-46fa-8a41-d63dfab6a47f-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:22:45.093755 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9d12dac5-c8d7-4cc0-801d-430781035787 tempest-VolumesBackupsTest-1744874988 None] Lock "cinder-2c387064-4e94-46fa-8a41-d63dfab6a47f-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:22:45.130320 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9d12dac5-c8d7-4cc0-801d-430781035787 tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:45.997323 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9d12dac5-c8d7-4cc0-801d-430781035787 tempest-VolumesBackupsTest-1744874988 None] Volume volume-2c387064-4e94-46fa-8a41-d63dfab6a47f is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:22:46.017549 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9d12dac5-c8d7-4cc0-801d-430781035787 tempest-VolumesBackupsTest-1744874988 None] deleting rbd volume volume-2c387064-4e94-46fa-8a41-d63dfab6a47f {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:22:46.311860 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-9d12dac5-c8d7-4cc0-801d-430781035787 tempest-VolumesBackupsTest-1744874988 None] Created reservations ['0fb4ff2b-81f1-4dc0-8ba4-874589d9ec78', 'cee2e5f6-a5a9-4af2-ab44-13cd48060db2', 'c4014b48-45cc-40a8-b5fe-f5d3c0c7f65d', '70589ce6-46db-430b-b113-4ac74f87d928'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:22:46.361940 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9d12dac5-c8d7-4cc0-801d-430781035787 tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:46.389856 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9d12dac5-c8d7-4cc0-801d-430781035787 tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:46.421221 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-9d12dac5-c8d7-4cc0-801d-430781035787 tempest-VolumesBackupsTest-1744874988 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:22:46.428461 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9d12dac5-c8d7-4cc0-801d-430781035787 tempest-VolumesBackupsTest-1744874988 None] Deleted volume successfully. Oct 01 21:22:46.449315 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9d12dac5-c8d7-4cc0-801d-430781035787 tempest-VolumesBackupsTest-1744874988 None] Lock "cinder-2c387064-4e94-46fa-8a41-d63dfab6a47f-delete_volume" released by "delete_volume" :: held 1.348s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:22:47.749859 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:22:47.771597 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Flow 'volume_create_manager' (8ad0f2cd-df55-49d3-b9d6-fb646764087d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:47.778968 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cfb9e11a-2515-4bb0-a0d9-b83314763290) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:47.799022 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cfb9e11a-2515-4bb0-a0d9-b83314763290) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:22:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-999261172',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=63348977-1730-47c3-b06d-d17c41516a30,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0bb47fed88494698a3be2a0b34761464',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:22:48Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:22:48Z,use_quota=True,user_id='2f6a9ae0c03d4328be8a4665cc503005',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:47.804532 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e18e7e88-2a91-4c95-8e72-49179bba9cfb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:47.808913 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e18e7e88-2a91-4c95-8e72-49179bba9cfb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:47.814175 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (30460279-133f-43e2-bac2-9cd9398dff19) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:47.905717 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (30460279-133f-43e2-bac2-9cd9398dff19) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '63348977-1730-47c3-b06d-d17c41516a30', 'volume_name': 'volume-63348977-1730-47c3-b06d-d17c41516a30', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:47.910817 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fcc4684f-b582-4237-8c8e-45b10ecad8dc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:47.915068 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fcc4684f-b582-4237-8c8e-45b10ecad8dc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:47.920111 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (57474123-398d-4593-80f3-7b4becf81802) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:47.921203 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Volume 63348977-1730-47c3-b06d-d17c41516a30: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-63348977-1730-47c3-b06d-d17c41516a30', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:22:47.921767 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Cloning 63348977-1730-47c3-b06d-d17c41516a30 from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:22:47.922608 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:22:47.931676 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:22:47.940857 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 22, 47, 936262)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:22:47.941298 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:22:47.941706 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:22:47.945883 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.014s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:22:47.947343 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:22:47.953500 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 22, 47, 950932)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:22:47.953993 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:22:47.954394 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:22:47.967712 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:48.035586 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] creating snapshot='volume-63348977-1730-47c3-b06d-d17c41516a30.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:22:48.499204 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-63348977-1730-47c3-b06d-d17c41516a30.clone_snap' to 'volume-63348977-1730-47c3-b06d-d17c41516a30' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:22:48.629344 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:22:48.630756 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:22:48.631141 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:22:48.649257 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:22:48.649914 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Marking volume 63348977-1730-47c3-b06d-d17c41516a30 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:22:48.659486 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to 63348977-1730-47c3-b06d-d17c41516a30. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:22:48.660007 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Creating volume glance metadata for volume 63348977-1730-47c3-b06d-d17c41516a30 backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:22:48.719607 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (57474123-398d-4593-80f3-7b4becf81802) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-63348977-1730-47c3-b06d-d17c41516a30', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:48.724151 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (70b7abf5-5f63-4a18-9c23-063762d955b5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:48.734152 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Volume volume-63348977-1730-47c3-b06d-d17c41516a30 (63348977-1730-47c3-b06d-d17c41516a30): created successfully Oct 01 21:22:48.738221 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (70b7abf5-5f63-4a18-9c23-063762d955b5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:48.744343 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Flow 'volume_create_manager' (8ad0f2cd-df55-49d3-b9d6-fb646764087d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:48.752546 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-37d4d72b-7877-484c-bdf2-b969af54371b tempest-VolumesBackupsTest-1744874988 None] Created volume successfully. Oct 01 21:22:48.754914 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=65,cinder:UPDATE=25,cinder:INSERT=10 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:22:50.083121 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9e5cd512-5cea-4ddf-8619-150077596eec tempest-VolumesBackupsTest-1744874988 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-10-01T21:22:49Z,data_timestamp=2022-10-01T21:22:49Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-2145151090',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=5ce842e4-7dda-4939-9346-f759166a0113,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='0bb47fed88494698a3be2a0b34761464',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='2f6a9ae0c03d4328be8a4665cc503005',volume_id=63348977-1730-47c3-b06d-d17c41516a30) Oct 01 21:22:50.083831 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.backup.rpcapi [None req-9e5cd512-5cea-4ddf-8619-150077596eec tempest-VolumesBackupsTest-1744874988 None] continue_backup in rpcapi backup_id 5ce842e4-7dda-4939-9346-f759166a0113 {{(pid=133391) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Oct 01 21:22:50.171041 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-9e5cd512-5cea-4ddf-8619-150077596eec tempest-VolumesBackupsTest-1744874988 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:22:50.941662 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-9e5cd512-5cea-4ddf-8619-150077596eec tempest-VolumesBackupsTest-1744874988 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.770s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:22:50.942810 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9e5cd512-5cea-4ddf-8619-150077596eec tempest-VolumesBackupsTest-1744874988 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-63348977-1730-47c3-b06d-d17c41516a30', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '63348977-1730-47c3-b06d-d17c41516a30', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:22:50.960245 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9e5cd512-5cea-4ddf-8619-150077596eec tempest-VolumesBackupsTest-1744874988 None] Initialize volume connection completed successfully. Oct 01 21:22:56.317348 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9e5cd512-5cea-4ddf-8619-150077596eec tempest-VolumesBackupsTest-1744874988 None] Terminate volume connection completed successfully. Oct 01 21:22:56.338200 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9e5cd512-5cea-4ddf-8619-150077596eec tempest-VolumesBackupsTest-1744874988 None] Remove volume export completed successfully. Oct 01 21:22:57.781959 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:22:57.804043 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Flow 'volume_create_manager' (a41516c0-8ac2-43ed-a2dc-dbafc501f9d5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:57.811415 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3c44be38-731e-42f8-8375-cb50b2eb3f23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:57.827646 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3c44be38-731e-42f8-8375-cb50b2eb3f23) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:22:58Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_5ce842e4-7dda-4939-9346-f759166a0113',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=af07c08d-779c-4030-ac05-a82b0e54f3f8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0bb47fed88494698a3be2a0b34761464',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:22:58Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:22:58Z,use_quota=True,user_id='2f6a9ae0c03d4328be8a4665cc503005',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:57.832793 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ce39b80b-a3d5-4961-b04d-7af663665ae2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:57.836800 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ce39b80b-a3d5-4961-b04d-7af663665ae2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:57.841855 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (beb20b85-b455-401a-b1ec-efe37eea8dd4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:57.845872 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (beb20b85-b455-401a-b1ec-efe37eea8dd4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'af07c08d-779c-4030-ac05-a82b0e54f3f8', 'volume_name': 'volume-af07c08d-779c-4030-ac05-a82b0e54f3f8', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:57.850602 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a36be014-a851-4968-a423-11b0093e75d6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:57.854527 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a36be014-a851-4968-a423-11b0093e75d6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:57.858996 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (78841ef1-2244-4e2b-862e-1a027fcb3814) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:57.860043 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Volume af07c08d-779c-4030-ac05-a82b0e54f3f8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-af07c08d-779c-4030-ac05-a82b0e54f3f8', 'volume_size': 1} Oct 01 21:22:57.860455 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] creating volume 'volume-af07c08d-779c-4030-ac05-a82b0e54f3f8' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:22:57.861572 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:22:57.918478 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:22:57.918939 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:22:57.922889 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (78841ef1-2244-4e2b-862e-1a027fcb3814) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-af07c08d-779c-4030-ac05-a82b0e54f3f8', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:57.927480 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (63331a46-d8a5-44c3-b5a1-5a7b0fd8d4a1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:22:57.938606 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Volume volume-af07c08d-779c-4030-ac05-a82b0e54f3f8 (af07c08d-779c-4030-ac05-a82b0e54f3f8): created successfully Oct 01 21:22:57.942670 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (63331a46-d8a5-44c3-b5a1-5a7b0fd8d4a1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:22:57.948935 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Flow 'volume_create_manager' (a41516c0-8ac2-43ed-a2dc-dbafc501f9d5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:22:57.977203 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Created volume successfully. Oct 01 21:22:58.007633 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-2d4c4616-fe2d-4929-86c5-2d8a3125b891 tempest-TestVolumeBootPattern-1545759599 None] Created reservations ['68b0266d-4f76-4b1b-8c3a-7d8985f8a3d1', '43407f4d-adfb-4a8a-89dd-5b2a561d559c', '8f301d8d-d2e4-4cf0-a939-3d9df11f96e5', '2a932ab0-d319-43c9-bf15-16dbfe5d3b46'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:22:58.057574 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-2d4c4616-fe2d-4929-86c5-2d8a3125b891 tempest-TestVolumeBootPattern-1545759599 None] Delete snapshot completed successfully. Oct 01 21:22:58.063047 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-2d4c4616-fe2d-4929-86c5-2d8a3125b891 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-67c87bce-6167-4219-9eee-20a0aeed139b-delete_snapshot" released by "delete_snapshot" :: held 30.254s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:22:58.818823 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:22:59.645081 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.826s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:22:59.646024 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-af07c08d-779c-4030-ac05-a82b0e54f3f8', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'af07c08d-779c-4030-ac05-a82b0e54f3f8', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:22:59.662220 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Initialize volume connection completed successfully. Oct 01 21:23:00.117811 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-a62e6fe0-ccd9-4ad3-a6a6-b4b12598b256 req-c556185d-a698-4eb5-b7d8-018c5b6ea6b7 tempest-TestVolumeBackupRestore-262965167 None] Terminate volume connection completed successfully. Oct 01 21:23:00.119209 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-a62e6fe0-ccd9-4ad3-a6a6-b4b12598b256 req-c556185d-a698-4eb5-b7d8-018c5b6ea6b7 tempest-TestVolumeBackupRestore-262965167 None] Deleting attachment 17f98f36-1a7c-472f-a5de-5c25e281de00. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:23:01.586156 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-f899c417-5ae6-497c-98a9-f321720d6792 req-7ba5a68a-c4a1-488f-946d-39175322e01e tempest-TestVolumeBootPattern-1545759599 None] Terminate volume connection completed successfully. Oct 01 21:23:01.588138 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-f899c417-5ae6-497c-98a9-f321720d6792 req-7ba5a68a-c4a1-488f-946d-39175322e01e tempest-TestVolumeBootPattern-1545759599 None] Deleting attachment 4570efd7-096c-45e9-a2cf-d065cae57707. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:23:01.822126 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-af3590a1-ce01-4f8a-a2f8-cac49fea20f7-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:01.845745 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-af3590a1-ce01-4f8a-a2f8-cac49fea20f7-delete_volume" acquired by "delete_volume" :: waited 0.024s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:01.964439 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:02.245483 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:23:02.316860 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] deleting rbd volume volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:02.328061 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6a3a2a4d-7061-4e67-8473-b214ee82ba89 tempest-TestVolumeBackupRestore-262965167 None] Acquiring lock "cinder-8f49ce01-740d-4958-95e8-e334234cf31f-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:02.331719 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] Acquiring lock "cinder-2c3c1619-9d87-4421-83b1-88c4fc46ebc4-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:02.355887 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6a3a2a4d-7061-4e67-8473-b214ee82ba89 tempest-TestVolumeBackupRestore-262965167 None] Lock "cinder-8f49ce01-740d-4958-95e8-e334234cf31f-delete_volume" acquired by "delete_volume" :: waited 0.028s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:02.361479 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] Lock "cinder-2c3c1619-9d87-4421-83b1-88c4fc46ebc4-delete_volume" acquired by "delete_volume" :: waited 0.030s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:02.465096 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6a3a2a4d-7061-4e67-8473-b214ee82ba89 tempest-TestVolumeBackupRestore-262965167 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:03.193092 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6a3a2a4d-7061-4e67-8473-b214ee82ba89 tempest-TestVolumeBackupRestore-262965167 None] Volume volume-8f49ce01-740d-4958-95e8-e334234cf31f is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:23:03.267985 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6a3a2a4d-7061-4e67-8473-b214ee82ba89 tempest-TestVolumeBackupRestore-262965167 None] deleting rbd volume volume-8f49ce01-740d-4958-95e8-e334234cf31f {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:03.314670 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:04.309158 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] deleting rbd volume volume-2c3c1619-9d87-4421-83b1-88c4fc46ebc4 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:04.313041 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:23:04.368042 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] deleting parent snapshot volume-af3590a1-ce01-4f8a-a2f8-cac49fea20f7.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:23:05.217103 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:23:05.254204 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] deleting parent snapshot volume-2c3c1619-9d87-4421-83b1-88c4fc46ebc4.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:23:06.218080 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=71,cinder:INSERT=10,cinder:UPDATE=17,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:06.265639 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Terminate volume connection completed successfully. Oct 01 21:23:06.299351 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0c187411-d2b6-46cf-a540-f30eb64ab625 tempest-VolumesBackupsTest-1744874988 None] Remove volume export completed successfully. Oct 01 21:23:06.328304 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] Created reservations ['e590c05e-8013-468d-af59-2102c77ac57b', 'dae73572-1985-4b75-8ded-bcf82bed2d07', '8cf1fec1-eea1-464f-9aed-19fea817af34', '070c672e-f87c-4e53-a3e8-1603a5da7d5a'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:06.343032 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-6a3a2a4d-7061-4e67-8473-b214ee82ba89 tempest-TestVolumeBackupRestore-262965167 None] Created reservations ['83330388-60eb-4773-859c-62961d288859', '4104a05b-4a12-4d8c-83d7-00d23affaeda', '60134e9f-efaa-4eb5-bd91-8c42aac831a0', 'c780f98e-6a85-4b01-8ba7-86decb502513'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:06.374175 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] Created reservations ['bb0c6b71-785e-45d2-b880-18718e444ce6', 'd7332673-24b6-4da0-935a-ac2566c66885', 'd9540d39-61f4-44f5-aaee-3ba63a79e042', 'e94564e9-6c97-4d92-8fd4-f28b25d6ddb1'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:06.419146 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=68,cinder:UPDATE=23,cinder:DELETE=1,cinder:INSERT=8 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:06.441176 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:06.478244 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6a3a2a4d-7061-4e67-8473-b214ee82ba89 tempest-TestVolumeBackupRestore-262965167 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:06.505846 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:06.538064 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:06.544539 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6a3a2a4d-7061-4e67-8473-b214ee82ba89 tempest-TestVolumeBackupRestore-262965167 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:06.572642 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-6a3a2a4d-7061-4e67-8473-b214ee82ba89 tempest-TestVolumeBackupRestore-262965167 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:06.580583 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6a3a2a4d-7061-4e67-8473-b214ee82ba89 tempest-TestVolumeBackupRestore-262965167 None] Deleted volume successfully. Oct 01 21:23:06.590351 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] Deleted volume successfully. Oct 01 21:23:06.606270 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6a3a2a4d-7061-4e67-8473-b214ee82ba89 tempest-TestVolumeBackupRestore-262965167 None] Lock "cinder-8f49ce01-740d-4958-95e8-e334234cf31f-delete_volume" released by "delete_volume" :: held 4.250s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:06.626022 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-f899c417-5ae6-497c-98a9-f321720d6792 req-29c34730-634f-42a9-978f-914fb1c33116 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-af3590a1-ce01-4f8a-a2f8-cac49fea20f7-delete_volume" released by "delete_volume" :: held 4.778s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:06.626413 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:06.675376 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:06.711638 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:06.717149 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] Deleted volume successfully. Oct 01 21:23:06.745568 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1804457c-2069-4dd8-bcd4-c0eb9459fb45 tempest-TestVolumeBackupRestore-262965167 None] Lock "cinder-2c3c1619-9d87-4421-83b1-88c4fc46ebc4-delete_volume" released by "delete_volume" :: held 4.371s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:07.334648 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c083f586-f158-4afd-acd0-770197e3b4b6 tempest-VolumesBackupsTest-1744874988 None] Acquiring lock "cinder-af07c08d-779c-4030-ac05-a82b0e54f3f8-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:07.354446 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c083f586-f158-4afd-acd0-770197e3b4b6 tempest-VolumesBackupsTest-1744874988 None] Lock "cinder-af07c08d-779c-4030-ac05-a82b0e54f3f8-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:07.410240 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c083f586-f158-4afd-acd0-770197e3b4b6 tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:08.288341 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c083f586-f158-4afd-acd0-770197e3b4b6 tempest-VolumesBackupsTest-1744874988 None] Volume volume-af07c08d-779c-4030-ac05-a82b0e54f3f8 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:23:08.310986 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c083f586-f158-4afd-acd0-770197e3b4b6 tempest-VolumesBackupsTest-1744874988 None] deleting rbd volume volume-af07c08d-779c-4030-ac05-a82b0e54f3f8 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:08.854708 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-c083f586-f158-4afd-acd0-770197e3b4b6 tempest-VolumesBackupsTest-1744874988 None] Created reservations ['6eb69f5b-a169-458d-b060-b3cb3e067482', 'f5e091a2-1ed1-4753-8624-681da8e171cf', '19b375d9-e46d-4afe-a3a6-151e794aa2f4', 'bbd429d7-3490-4f59-b548-0749202acdae'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:08.925480 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c083f586-f158-4afd-acd0-770197e3b4b6 tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:08.975581 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c083f586-f158-4afd-acd0-770197e3b4b6 tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:09.042548 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c083f586-f158-4afd-acd0-770197e3b4b6 tempest-VolumesBackupsTest-1744874988 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:09.050124 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c083f586-f158-4afd-acd0-770197e3b4b6 tempest-VolumesBackupsTest-1744874988 None] Deleted volume successfully. Oct 01 21:23:09.071182 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c083f586-f158-4afd-acd0-770197e3b4b6 tempest-VolumesBackupsTest-1744874988 None] Lock "cinder-af07c08d-779c-4030-ac05-a82b0e54f3f8-delete_volume" released by "delete_volume" :: held 1.718s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:11.068537 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:23:11.119514 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (c3f8cae9-745c-4df1-a14a-973a622972eb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:11.127823 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c02b6aaf-f85f-46ce-978c-84ba392833cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:11.146959 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c02b6aaf-f85f-46ce-978c-84ba392833cc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:23:11Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1019511945',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=91a33031-458c-4c6f-8fe0-fccc855ffd7f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='df0e56b020444dd28af868590b1bccf4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:23:11Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:23:11Z,use_quota=True,user_id='eb0bf5fce885439eaa16a0871c2992ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:11.153610 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ae8dc3ae-1c3d-4c11-b9e0-6358fca49e8c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:11.158458 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ae8dc3ae-1c3d-4c11-b9e0-6358fca49e8c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:11.163420 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (46a7e4f3-c070-4e37-8d3d-44b16aa7da18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:11.278787 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (46a7e4f3-c070-4e37-8d3d-44b16aa7da18) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'volume_name': 'volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:11.283628 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a805b84-cc91-407d-b969-f953d0df7a87) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:11.287447 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a805b84-cc91-407d-b969-f953d0df7a87) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:11.291976 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (000a567f-61df-43b8-b4a4-430aae89bf8d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:11.293326 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Volume 91a33031-458c-4c6f-8fe0-fccc855ffd7f: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:23:11.293895 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Cloning 91a33031-458c-4c6f-8fe0-fccc855ffd7f from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:23:11.294668 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:11.303792 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:11.314243 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 23, 11, 308765)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:23:11.314762 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:23:11.315168 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:23:11.323247 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.019s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:11.324236 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:23:11.331247 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 23, 11, 328692)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:23:11.332197 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:23:11.332702 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:23:11.346943 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:11.437848 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] creating snapshot='volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:23:12.404926 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f.clone_snap' to 'volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:23:12.411095 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] Acquiring lock "cinder-63348977-1730-47c3-b06d-d17c41516a30-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:12.433401 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] Lock "cinder-63348977-1730-47c3-b06d-d17c41516a30-delete_volume" acquired by "delete_volume" :: waited 0.023s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:12.509502 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:13.460923 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] deleting rbd volume volume-63348977-1730-47c3-b06d-d17c41516a30 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:13.526040 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:23:13.527959 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:23:13.528335 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:23:13.584650 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:23:13.588794 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Marking volume 91a33031-458c-4c6f-8fe0-fccc855ffd7f as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:23:13.643036 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to 91a33031-458c-4c6f-8fe0-fccc855ffd7f. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:23:13.643649 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Creating volume glance metadata for volume 91a33031-458c-4c6f-8fe0-fccc855ffd7f backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:23:13.690013 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=51,cinder:INSERT=10,cinder:UPDATE=37,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:13.742410 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (000a567f-61df-43b8-b4a4-430aae89bf8d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:13.751265 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7289ddb7-04ae-40b5-aa1c-2754ff19f81e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:13.768735 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Volume volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f (91a33031-458c-4c6f-8fe0-fccc855ffd7f): created successfully Oct 01 21:23:13.775836 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7289ddb7-04ae-40b5-aa1c-2754ff19f81e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:13.787062 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (c3f8cae9-745c-4df1-a14a-973a622972eb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:13.805127 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-11c2a637-3205-4539-9871-9276453f0b21 tempest-TestVolumeBootPattern-1545759599 None] Created volume successfully. Oct 01 21:23:13.817187 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:23:13.857865 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] deleting parent snapshot volume-63348977-1730-47c3-b06d-d17c41516a30.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:23:14.515880 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] Created reservations ['e7ac15f6-6665-43de-8c64-263f0d9aac85', '0d9a7430-a010-49ac-b4f9-20794c81083b', 'eba82909-aac2-463e-a740-2ab59ba7db4b', '184eb86f-643b-48c9-8f0c-a0b15a1871fb'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:14.571497 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:14.613393 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:14.659181 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:14.665816 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] Deleted volume successfully. Oct 01 21:23:14.686053 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3df8c26b-b078-45d6-b9f9-b8b549c300fd tempest-VolumesBackupsTest-1744874988 None] Lock "cinder-63348977-1730-47c3-b06d-d17c41516a30-delete_volume" released by "delete_volume" :: held 2.253s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:17.718939 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-5210f1ba-7b79-4d14-884e-2601297e9c90 req-9c834558-a66f-4d04-ba17-97b8a01dd9f7 tempest-TestVolumeBootPattern-1545759599 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:23:18.434107 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:23:18.476754 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (6890a808-8f24-4427-b38e-e8c9ca0d414b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:18.483853 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (04ff5221-4be8-4cc3-97b2-12e54134044c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:18.504532 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (04ff5221-4be8-4cc3-97b2-12e54134044c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:23:18Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1769735104',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=994d7a73-14e7-4bb8-b18a-9e7547a9aa90,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='243e5655a8a04805942d9fe4603bf8e5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:23:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:23:18Z,use_quota=True,user_id='c8d29756718f45c0a8a715a00daa1505',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:18.511317 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1833298b-3432-4a6a-afc8-86e979480708) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:18.516138 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1833298b-3432-4a6a-afc8-86e979480708) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:18.522017 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d96c09d3-1788-4c58-a592-2334a521483c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:18.530716 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d96c09d3-1788-4c58-a592-2334a521483c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '994d7a73-14e7-4bb8-b18a-9e7547a9aa90', 'volume_name': 'volume-994d7a73-14e7-4bb8-b18a-9e7547a9aa90', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:18.535923 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f24e2aab-de22-44b2-88f9-e90d869d428c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:18.540609 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f24e2aab-de22-44b2-88f9-e90d869d428c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:18.546872 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0499c166-9268-4bc4-89ca-05f396584148) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:18.548082 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Volume 994d7a73-14e7-4bb8-b18a-9e7547a9aa90: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-994d7a73-14e7-4bb8-b18a-9e7547a9aa90', 'volume_size': 1} Oct 01 21:23:18.548582 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] creating volume 'volume-994d7a73-14e7-4bb8-b18a-9e7547a9aa90' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:23:18.549830 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:18.660617 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:23:18.661207 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:23:18.665266 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0499c166-9268-4bc4-89ca-05f396584148) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-994d7a73-14e7-4bb8-b18a-9e7547a9aa90', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:18.674903 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5347c8f6-cc46-45b1-ac34-9dbcdd615b64) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:18.701315 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Volume volume-994d7a73-14e7-4bb8-b18a-9e7547a9aa90 (994d7a73-14e7-4bb8-b18a-9e7547a9aa90): created successfully Oct 01 21:23:18.706200 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5347c8f6-cc46-45b1-ac34-9dbcdd615b64) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:18.713561 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (6890a808-8f24-4427-b38e-e8c9ca0d414b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:18.742850 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f7623418-b4ea-4a62-9863-67407f4cab13 tempest-VolumesGetTest-1519786071 None] Created volume successfully. Oct 01 21:23:18.936756 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-5210f1ba-7b79-4d14-884e-2601297e9c90 req-9c834558-a66f-4d04-ba17-97b8a01dd9f7 tempest-TestVolumeBootPattern-1545759599 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.218s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:23:18.937522 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-5210f1ba-7b79-4d14-884e-2601297e9c90 req-9c834558-a66f-4d04-ba17-97b8a01dd9f7 tempest-TestVolumeBootPattern-1545759599 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:23:18.961489 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-5210f1ba-7b79-4d14-884e-2601297e9c90 req-9c834558-a66f-4d04-ba17-97b8a01dd9f7 tempest-TestVolumeBootPattern-1545759599 None] Connection info returned from driver {'name': 'volumes/volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'c79719d8-75ae-45fa-b6ed-87600ec992f0'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:23:19.013381 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=53,cinder:DELETE=3,cinder:UPDATE=30,cinder:INSERT=14 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:19.033881 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-5210f1ba-7b79-4d14-884e-2601297e9c90 req-9c834558-a66f-4d04-ba17-97b8a01dd9f7 tempest-TestVolumeBootPattern-1545759599 None] attachment_update completed successfully. Oct 01 21:23:20.293513 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:23:20.316101 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (3c3713f5-df29-49e3-803a-2c48f24d74f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:20.323702 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9e8567f6-5dd1-4732-bcb1-5335dc8d135a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:20.341577 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9e8567f6-5dd1-4732-bcb1-5335dc8d135a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:23:20Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-550681275',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1f2ddfe1-d588-496d-954c-338d8ee37938,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='243e5655a8a04805942d9fe4603bf8e5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:23:20Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:23:20Z,use_quota=True,user_id='c8d29756718f45c0a8a715a00daa1505',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:20.346730 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e4acce93-0d19-4997-83d9-e3602f47acbb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:20.350770 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e4acce93-0d19-4997-83d9-e3602f47acbb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:20.356318 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ca401648-553c-48ab-8828-5b0bc15bf38d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:20.360245 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ca401648-553c-48ab-8828-5b0bc15bf38d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1f2ddfe1-d588-496d-954c-338d8ee37938', 'volume_name': 'volume-1f2ddfe1-d588-496d-954c-338d8ee37938', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:20.364990 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0ea97253-e092-4e87-851f-58891ab820c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:20.368972 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0ea97253-e092-4e87-851f-58891ab820c7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:20.373986 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4c9dfbde-40c1-4d29-bfe0-5039d93a9240) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:20.375288 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Volume 1f2ddfe1-d588-496d-954c-338d8ee37938: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1f2ddfe1-d588-496d-954c-338d8ee37938', 'volume_size': 1} Oct 01 21:23:20.375924 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] creating volume 'volume-1f2ddfe1-d588-496d-954c-338d8ee37938' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:23:20.376760 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:20.480805 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:23:20.481879 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:23:20.487012 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4c9dfbde-40c1-4d29-bfe0-5039d93a9240) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1f2ddfe1-d588-496d-954c-338d8ee37938', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:20.492101 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (34e555cf-2bfc-4a00-a502-33c8957ff523) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:20.503390 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Volume volume-1f2ddfe1-d588-496d-954c-338d8ee37938 (1f2ddfe1-d588-496d-954c-338d8ee37938): created successfully Oct 01 21:23:20.508151 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (34e555cf-2bfc-4a00-a502-33c8957ff523) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:20.514814 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (3c3713f5-df29-49e3-803a-2c48f24d74f2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:20.525249 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7f468ace-fdf4-43c2-9092-f1885d80460e tempest-VolumesGetTest-1519786071 None] Created volume successfully. Oct 01 21:23:21.492725 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] Acquiring lock "cinder-1f2ddfe1-d588-496d-954c-338d8ee37938-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:21.520314 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] Lock "cinder-1f2ddfe1-d588-496d-954c-338d8ee37938-delete_volume" acquired by "delete_volume" :: waited 0.028s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:21.563822 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:21.629291 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:23:21.629794 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] Volume volume-1f2ddfe1-d588-496d-954c-338d8ee37938 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:23:21.634563 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] deleting rbd volume volume-1f2ddfe1-d588-496d-954c-338d8ee37938 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:21.880215 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] Created reservations ['12d7aa9a-3fc7-4e62-8495-82c4c0ad382c', 'd11754b5-0956-4bed-9523-778455966331', '67dce483-4866-45b6-a960-bca14e4a1269', 'edac8e8d-6dd7-4d9f-ba14-5c7dd1840d50'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:21.926404 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:21.952485 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:22.008709 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:22.018614 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] Deleted volume successfully. Oct 01 21:23:22.031790 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-5bb0833a-d948-403c-ab2e-48779e4d0714 tempest-VolumesGetTest-1519786071 None] Lock "cinder-1f2ddfe1-d588-496d-954c-338d8ee37938-delete_volume" released by "delete_volume" :: held 0.511s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:22.706760 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] Acquiring lock "cinder-994d7a73-14e7-4bb8-b18a-9e7547a9aa90-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:22.720429 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] Lock "cinder-994d7a73-14e7-4bb8-b18a-9e7547a9aa90-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:22.841842 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:22.899787 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:23:22.900344 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] Volume volume-994d7a73-14e7-4bb8-b18a-9e7547a9aa90 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:23:22.905647 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] deleting rbd volume volume-994d7a73-14e7-4bb8-b18a-9e7547a9aa90 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:23.105141 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] Created reservations ['65958f43-326d-4fac-b8a6-d8fa2fe2667e', '9da6b56e-ac08-4307-bda1-5b2555739000', '97b67f17-9386-4663-8b06-1e6ec8b1837d', '1f826129-694d-412d-976c-eaf58ff7351d'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:23.151805 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:23.181555 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:23.208043 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:23.214098 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] Deleted volume successfully. Oct 01 21:23:23.227114 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3a0f20fa-1e21-4b4c-90da-ce3ebbfeb5e0 tempest-VolumesGetTest-1519786071 None] Lock "cinder-994d7a73-14e7-4bb8-b18a-9e7547a9aa90-delete_volume" released by "delete_volume" :: held 0.507s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:24.203949 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:23:24.223044 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (9783baea-1336-4219-b37a-a51771deeab1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:24.229364 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5fbd2a6c-85c7-461a-88ca-f911ff1dac95) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:24.235143 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=61,cinder:UPDATE=28,cinder:DELETE=4,cinder:INSERT=7 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:24.247125 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5fbd2a6c-85c7-461a-88ca-f911ff1dac95) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:23:24Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1402957237',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='243e5655a8a04805942d9fe4603bf8e5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:23:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:23:24Z,use_quota=True,user_id='c8d29756718f45c0a8a715a00daa1505',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:24.252582 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (72033a27-66dd-4c98-8e86-4a2257247860) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:24.256817 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (72033a27-66dd-4c98-8e86-4a2257247860) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:24.262445 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (772f5dff-25b7-4549-88d7-2f428a9af747) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:24.266581 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (772f5dff-25b7-4549-88d7-2f428a9af747) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a', 'volume_name': 'volume-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:24.271510 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (05e84dcc-0473-4ace-94ac-2650f19bafcb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:24.275652 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (05e84dcc-0473-4ace-94ac-2650f19bafcb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:24.280361 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e941f420-c87b-4a18-b99e-1cdb613510f1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:24.281546 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Volume 53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a', 'volume_size': 1} Oct 01 21:23:24.282012 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] creating volume 'volume-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:23:24.282892 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:24.355768 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:23:24.356213 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:23:24.360398 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e941f420-c87b-4a18-b99e-1cdb613510f1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:24.367062 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e0a5dcd0-448c-4a68-95d8-4e9af681668a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:24.378283 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Volume volume-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a (53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a): created successfully Oct 01 21:23:24.384252 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e0a5dcd0-448c-4a68-95d8-4e9af681668a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:24.391112 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (9783baea-1336-4219-b37a-a51771deeab1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:24.400678 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8fceea44-0060-40b2-bc48-f2a761aa4e5f tempest-VolumesGetTest-1519786071 None] Created volume successfully. Oct 01 21:23:25.599457 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:23:25.632108 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (023bc12c-3d76-414d-9379-b652cf9fae6d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:25.640759 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2e500262-5a81-4ce5-9d47-f29e725cd7ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:25.659432 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2e500262-5a81-4ce5-9d47-f29e725cd7ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:23:25Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-413542194',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=608e986c-d8fe-430b-bbc4-9da03c59195c,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='243e5655a8a04805942d9fe4603bf8e5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:23:26Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a,status='creating',terminated_at=None,updated_at=2022-10-01T21:23:26Z,use_quota=True,user_id='c8d29756718f45c0a8a715a00daa1505',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:25.666025 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (163a1a4e-0a51-49d0-a407-ece7cd6b09fc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:25.670975 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (163a1a4e-0a51-49d0-a407-ece7cd6b09fc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:25.675831 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bc24b999-8bc7-4583-badc-cddb697af541) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:25.692904 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bc24b999-8bc7-4583-badc-cddb697af541) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '608e986c-d8fe-430b-bbc4-9da03c59195c', 'volume_name': 'volume-608e986c-d8fe-430b-bbc4-9da03c59195c', 'volume_size': 1, 'source_volid': '53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a', 'source_volstatus': 'available'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:25.697632 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b737b5f6-d337-4cd8-a68f-ca6a50dd4be3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:25.701389 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b737b5f6-d337-4cd8-a68f-ca6a50dd4be3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:25.705833 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (72fd5279-7270-43a1-8428-2ef818c2e697) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:25.706895 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Volume 608e986c-d8fe-430b-bbc4-9da03c59195c: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-608e986c-d8fe-430b-bbc4-9da03c59195c', 'volume_size': 1, 'source_volid': '53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a', 'source_volstatus': 'available'} Oct 01 21:23:25.719362 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:25.758089 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] creating snapshot='volume-608e986c-d8fe-430b-bbc4-9da03c59195c.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:23:26.088614 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] cloning 'volume-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a@volume-608e986c-d8fe-430b-bbc4-9da03c59195c.clone_snap' to 'volume-608e986c-d8fe-430b-bbc4-9da03c59195c' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:23:26.187333 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Volume volume-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:23:26.188620 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:23:26.189003 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:23:26.217081 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:23:26.221382 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (72fd5279-7270-43a1-8428-2ef818c2e697) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-608e986c-d8fe-430b-bbc4-9da03c59195c', 'volume_size': 1, 'source_volid': '53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a', 'source_volstatus': 'available'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:26.230185 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (370d2b1f-007d-4f82-a52e-72f31ffca9eb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:26.253322 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Volume volume-608e986c-d8fe-430b-bbc4-9da03c59195c (608e986c-d8fe-430b-bbc4-9da03c59195c): created successfully Oct 01 21:23:26.262752 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (370d2b1f-007d-4f82-a52e-72f31ffca9eb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:26.268138 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (023bc12c-3d76-414d-9379-b652cf9fae6d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:26.284002 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-87f298f8-2078-4ca8-82aa-291ca6e9004f tempest-VolumesGetTest-1519786071 None] Created volume successfully. Oct 01 21:23:27.284006 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:23:27.306743 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (209fc4a5-80bf-485c-9c03-fa42029955e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:27.313039 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (307d6c90-988b-4870-8273-9254bd050fd2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:27.332389 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (307d6c90-988b-4870-8273-9254bd050fd2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:23:27Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-1640193938',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4560954e-18e2-4796-aa77-520561732c14,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='243e5655a8a04805942d9fe4603bf8e5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:23:27Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:23:27Z,use_quota=True,user_id='c8d29756718f45c0a8a715a00daa1505',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:27.338479 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (44ba70f8-0eca-44dd-ad10-ea5ded317414) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:27.342709 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (44ba70f8-0eca-44dd-ad10-ea5ded317414) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:27.348042 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bf3a8033-02b4-4021-90eb-04f661367d02) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:27.352679 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bf3a8033-02b4-4021-90eb-04f661367d02) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4560954e-18e2-4796-aa77-520561732c14', 'volume_name': 'volume-4560954e-18e2-4796-aa77-520561732c14', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:27.358010 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (95fef9ef-1511-4882-b2a4-bc357e56f41f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:27.361966 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (95fef9ef-1511-4882-b2a4-bc357e56f41f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:27.366714 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2bae203d-0e34-458c-abbe-055540ed1ff6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:27.368250 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Volume 4560954e-18e2-4796-aa77-520561732c14: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4560954e-18e2-4796-aa77-520561732c14', 'volume_size': 1} Oct 01 21:23:27.368653 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] creating volume 'volume-4560954e-18e2-4796-aa77-520561732c14' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:23:27.369314 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:27.449222 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:23:27.449752 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:23:27.471547 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2bae203d-0e34-458c-abbe-055540ed1ff6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4560954e-18e2-4796-aa77-520561732c14', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:27.476020 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bf49be8f-d439-4f0e-9839-9078178fff4b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:27.485817 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Volume volume-4560954e-18e2-4796-aa77-520561732c14 (4560954e-18e2-4796-aa77-520561732c14): created successfully Oct 01 21:23:27.489544 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bf49be8f-d439-4f0e-9839-9078178fff4b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:27.495027 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (209fc4a5-80bf-485c-9c03-fa42029955e5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:27.504381 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3500a8db-a275-4990-96a5-dad8b524eee9 tempest-VolumesGetTest-1519786071 None] Created volume successfully. Oct 01 21:23:28.402818 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] Acquiring lock "cinder-4560954e-18e2-4796-aa77-520561732c14-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:28.412545 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] Lock "cinder-4560954e-18e2-4796-aa77-520561732c14-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:28.446912 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:28.490670 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:23:28.491090 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] Volume volume-4560954e-18e2-4796-aa77-520561732c14 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:23:28.495607 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] deleting rbd volume volume-4560954e-18e2-4796-aa77-520561732c14 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:28.694203 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] Created reservations ['b37bde1c-0378-498d-b95d-6760800204e7', 'da68a337-c39e-4fd2-9aa6-3652c832ee41', '7bd227b8-6874-4ac1-b0d2-d4966e1b4752', '086bd499-f035-44b8-87aa-f59768a0c6f5'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:28.736786 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:28.760746 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:28.786794 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:28.791347 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] Deleted volume successfully. Oct 01 21:23:28.805545 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4448e225-7186-4d78-acad-e7a83faede7e tempest-VolumesGetTest-1519786071 None] Lock "cinder-4560954e-18e2-4796-aa77-520561732c14-delete_volume" released by "delete_volume" :: held 0.393s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:29.546817 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] Acquiring lock "cinder-608e986c-d8fe-430b-bbc4-9da03c59195c-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:29.561314 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] Lock "cinder-608e986c-d8fe-430b-bbc4-9da03c59195c-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:29.620958 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:29.701054 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:23:29.708827 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] deleting rbd volume volume-608e986c-d8fe-430b-bbc4-9da03c59195c {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:29.932144 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:23:29.951518 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] Volume volume-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:23:29.952188 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] deleting parent snapshot volume-608e986c-d8fe-430b-bbc4-9da03c59195c.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:23:30.184328 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=60,cinder:UPDATE=30,cinder:INSERT=6,cinder:DELETE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:30.215079 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] Created reservations ['fa99fbd1-4b15-4936-a96a-dd89030321a3', 'e3fdc546-e1ef-41ab-b8fd-204fb0009fa5', '7a0952fb-841a-45bd-9053-6c072ae7a14d', '7cf5fd10-35d0-4a1d-8d04-2a736bfc4ac2'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:30.268201 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:30.328924 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:30.379593 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:30.383994 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] Deleted volume successfully. Oct 01 21:23:30.400418 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ea4aaf84-7365-4bc5-9160-f353862140d0 tempest-VolumesGetTest-1519786071 None] Lock "cinder-608e986c-d8fe-430b-bbc4-9da03c59195c-delete_volume" released by "delete_volume" :: held 0.839s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:31.154616 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:23:31.173595 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (cc938435-e8d3-4af0-a8e0-35f4a877391a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:31.180044 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (981523c6-1269-4b4f-9f64-49941ad62a03) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:31.196871 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (981523c6-1269-4b4f-9f64-49941ad62a03) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:23:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-746674096',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=337b7076-1f94-457e-a8a1-d14237cc8f4d,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='243e5655a8a04805942d9fe4603bf8e5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:23:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:23:31Z,use_quota=True,user_id='c8d29756718f45c0a8a715a00daa1505',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:31.202732 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (343185ab-219e-4d12-8b96-c608e2d03766) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:31.207065 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (343185ab-219e-4d12-8b96-c608e2d03766) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:31.212668 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23682a32-3ecc-4683-9180-4f846291352b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:31.294585 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23682a32-3ecc-4683-9180-4f846291352b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '337b7076-1f94-457e-a8a1-d14237cc8f4d', 'volume_name': 'volume-337b7076-1f94-457e-a8a1-d14237cc8f4d', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:31.299515 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d77ee9de-9904-42e2-b2fe-14da27b2fe93) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:31.303551 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d77ee9de-9904-42e2-b2fe-14da27b2fe93) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:31.308079 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6682dbdc-01c1-45d9-a5a1-242b6d130a50) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:31.309155 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Volume 337b7076-1f94-457e-a8a1-d14237cc8f4d: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-337b7076-1f94-457e-a8a1-d14237cc8f4d', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:23:31.309726 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Cloning 337b7076-1f94-457e-a8a1-d14237cc8f4d from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:23:31.310466 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:31.319932 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:31.330062 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 23, 31, 324399)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:23:31.330589 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:23:31.331001 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:23:31.335351 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.016s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:31.336847 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:23:31.342625 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 23, 31, 340295)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:23:31.343038 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:23:31.343424 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:23:31.356377 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:31.409863 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] creating snapshot='volume-337b7076-1f94-457e-a8a1-d14237cc8f4d.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:23:32.195300 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-337b7076-1f94-457e-a8a1-d14237cc8f4d.clone_snap' to 'volume-337b7076-1f94-457e-a8a1-d14237cc8f4d' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:23:32.353440 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:23:32.360654 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:23:32.361040 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:23:32.393186 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:23:32.393959 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Marking volume 337b7076-1f94-457e-a8a1-d14237cc8f4d as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:23:32.407578 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to 337b7076-1f94-457e-a8a1-d14237cc8f4d. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:23:32.408039 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Creating volume glance metadata for volume 337b7076-1f94-457e-a8a1-d14237cc8f4d backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:23:32.498077 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6682dbdc-01c1-45d9-a5a1-242b6d130a50) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-337b7076-1f94-457e-a8a1-d14237cc8f4d', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:32.503975 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0402c091-3fe7-47f0-a21b-199eb5d9efed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:32.516384 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Volume volume-337b7076-1f94-457e-a8a1-d14237cc8f4d (337b7076-1f94-457e-a8a1-d14237cc8f4d): created successfully Oct 01 21:23:32.521051 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0402c091-3fe7-47f0-a21b-199eb5d9efed) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:32.527487 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (cc938435-e8d3-4af0-a8e0-35f4a877391a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:32.539022 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-75f74356-560c-446e-857c-e2b13bc8b0e8 tempest-VolumesGetTest-1519786071 None] Created volume successfully. Oct 01 21:23:32.545169 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=57,cinder:UPDATE=29,cinder:INSERT=12,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:33.782981 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:23:33.802305 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (d3735408-d39a-494f-b555-fd8e0d85b099) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:33.808430 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2f829844-8fa0-4edb-870a-752aeb143357) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:33.823544 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2f829844-8fa0-4edb-870a-752aeb143357) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:23:34Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-375058472',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5040f408-995e-40c0-a917-9340add57543,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='243e5655a8a04805942d9fe4603bf8e5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:23:34Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:23:34Z,use_quota=True,user_id='c8d29756718f45c0a8a715a00daa1505',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:33.828585 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (513223c8-4c3a-45b5-b636-53f8996aed3d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:33.832891 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (513223c8-4c3a-45b5-b636-53f8996aed3d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:33.838270 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3bde7060-bafb-4e1b-af40-41bea126b617) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:33.842797 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3bde7060-bafb-4e1b-af40-41bea126b617) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5040f408-995e-40c0-a917-9340add57543', 'volume_name': 'volume-5040f408-995e-40c0-a917-9340add57543', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:33.847864 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (31eb37ee-2e05-4ba8-a8a7-5b78adf19f08) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:33.851696 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (31eb37ee-2e05-4ba8-a8a7-5b78adf19f08) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:33.856730 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8eb2aa8-32a0-4853-9117-f36761daa0e2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:33.857905 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Volume 5040f408-995e-40c0-a917-9340add57543: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5040f408-995e-40c0-a917-9340add57543', 'volume_size': 1} Oct 01 21:23:33.858297 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] creating volume 'volume-5040f408-995e-40c0-a917-9340add57543' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:23:33.859083 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:33.920171 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:23:33.920823 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:23:33.925746 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8eb2aa8-32a0-4853-9117-f36761daa0e2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5040f408-995e-40c0-a917-9340add57543', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:33.931429 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6a7655f6-8ecf-4556-9c1d-4036c9a436d6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:33.940266 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Volume volume-5040f408-995e-40c0-a917-9340add57543 (5040f408-995e-40c0-a917-9340add57543): created successfully Oct 01 21:23:33.943830 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6a7655f6-8ecf-4556-9c1d-4036c9a436d6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:33.949051 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Flow 'volume_create_manager' (d3735408-d39a-494f-b555-fd8e0d85b099) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:33.956760 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-fc9cf86f-3cf8-42f6-9972-9d267f2e7201 tempest-VolumesGetTest-1519786071 None] Created volume successfully. Oct 01 21:23:34.899352 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] Acquiring lock "cinder-5040f408-995e-40c0-a917-9340add57543-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:34.910762 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] Lock "cinder-5040f408-995e-40c0-a917-9340add57543-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:34.946241 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:35.005251 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:23:35.005782 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] Volume volume-5040f408-995e-40c0-a917-9340add57543 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:23:35.011187 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] deleting rbd volume volume-5040f408-995e-40c0-a917-9340add57543 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:35.217431 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] Created reservations ['16eac7e8-8a3a-4eaf-9221-d8a37dd2f3e6', 'bde441d8-5ec4-4cc9-910e-c41823bd231a', '3b254a81-9a8e-46cf-b86c-b002be255065', '52160107-79c4-4692-af4d-86a86bbc02ab'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:35.258371 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:35.282450 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:35.307122 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:35.311240 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] Deleted volume successfully. Oct 01 21:23:35.328467 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-aab860de-edf6-4f3f-b7bd-d01572303ae3 tempest-VolumesGetTest-1519786071 None] Lock "cinder-5040f408-995e-40c0-a917-9340add57543-delete_volume" released by "delete_volume" :: held 0.413s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:36.058045 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] Acquiring lock "cinder-337b7076-1f94-457e-a8a1-d14237cc8f4d-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:36.068780 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] Lock "cinder-337b7076-1f94-457e-a8a1-d14237cc8f4d-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:36.109485 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:36.161922 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:23:36.169304 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] deleting rbd volume volume-337b7076-1f94-457e-a8a1-d14237cc8f4d {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:36.308399 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:23:36.339624 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] deleting parent snapshot volume-337b7076-1f94-457e-a8a1-d14237cc8f4d.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:23:37.251346 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] Created reservations ['b5434b6f-12c9-4167-86c6-6fb675c94931', '179012d1-8c4e-4730-9526-a3e34cad7a97', '9d6bb4a6-f70c-456b-93f3-71830eb5f25c', 'f8d7b5e5-a821-44b9-99f2-68446a7d06ce'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:37.296746 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:37.320409 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:37.345568 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:37.349752 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] Deleted volume successfully. Oct 01 21:23:37.361437 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d23bcb8f-b7c8-40b4-a4a9-e141387f5d59 tempest-VolumesGetTest-1519786071 None] Lock "cinder-337b7076-1f94-457e-a8a1-d14237cc8f4d-delete_volume" released by "delete_volume" :: held 1.293s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:38.250828 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] Acquiring lock "cinder-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:23:38.262149 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] Lock "cinder-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:23:38.295646 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:38.343633 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:23:38.344063 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] Volume volume-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:23:38.348530 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] deleting rbd volume volume-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:23:38.572928 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] Created reservations ['c578f0b0-5dce-4763-b0d0-c8395dfc3443', '68de501f-76c5-4d8f-9482-d7acdf8d0bdb', '0edce4e5-9eab-4632-bbdd-de155896ff68', '4aea1b91-a807-4d67-8ffb-6600aa249e62'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:23:38.617012 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=31,cinder:SELECT=56,cinder:INSERT=9,cinder:DELETE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:23:38.622982 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:38.650072 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:38.675845 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:23:38.680242 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] Deleted volume successfully. Oct 01 21:23:38.691738 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-fc8dee8a-e013-4e7b-895d-d4a6b7a6e5ee tempest-VolumesGetTest-1519786071 None] Lock "cinder-53904b1e-7ae7-4a0b-a2c9-3c323beb4f4a-delete_volume" released by "delete_volume" :: held 0.430s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:23:41.661325 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:23:41.697507 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Flow 'volume_create_manager' (608778d5-7732-4882-ba28-82a3784b23d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:41.704112 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (dbc15d0a-fe91-40b8-8840-039b454ff1ac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:41.722643 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (dbc15d0a-fe91-40b8-8840-039b454ff1ac) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:23:41Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesNegativeTest-Volume-1466170808',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3702ba8854c041f4ad1a3de3347dd5b3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:23:42Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:23:42Z,use_quota=True,user_id='fb9b3c506e6244b28569e68fb36ad5fc',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:41.727864 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (035cb1e7-06d1-46ac-b620-8f3b0b8e3ee2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:41.731826 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (035cb1e7-06d1-46ac-b620-8f3b0b8e3ee2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:41.736821 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8e668849-97bc-4cff-b7c5-770c0d3e772f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:41.740823 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8e668849-97bc-4cff-b7c5-770c0d3e772f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd4c67cfb-4b2e-4f8e-a16a-fcbc8b185503', 'volume_name': 'volume-d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:41.745610 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9add13c2-8ff0-4322-ba20-bb286db2fb6f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:41.749351 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9add13c2-8ff0-4322-ba20-bb286db2fb6f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:41.753857 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1f5313fc-fbfa-4699-89d7-4f5117265c47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:41.754924 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Volume d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503', 'volume_size': 1} Oct 01 21:23:41.755310 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] creating volume 'volume-d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:23:41.756228 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:23:41.814825 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:23:41.815116 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:23:41.818700 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1f5313fc-fbfa-4699-89d7-4f5117265c47) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:41.823160 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c1bddb3f-b7e1-48fb-a7eb-159bebfc81f0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:23:41.833278 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Volume volume-d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503 (d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503): created successfully Oct 01 21:23:41.837432 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c1bddb3f-b7e1-48fb-a7eb-159bebfc81f0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:23:41.843405 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Flow 'volume_create_manager' (608778d5-7732-4882-ba28-82a3784b23d7) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:23:41.855704 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a4a54498-4fcf-4619-9721-59de9b2525a6 tempest-VolumesNegativeTest-514173341 None] Created volume successfully. Oct 01 21:24:03.384037 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] Acquiring lock "cinder-d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:24:03.394306 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] Lock "cinder-d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:24:03.398803 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=53,cinder:UPDATE=29,cinder:INSERT=8,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:03.439368 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:03.483220 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:24:03.483798 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] Volume volume-d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:24:03.493847 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] deleting rbd volume volume-d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:24:03.765805 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] Created reservations ['78073238-5ae4-41eb-a321-32fd7712cdf3', 'c722d0d1-55fa-447b-ad15-95019b25ce98', '5768a410-768b-4595-9a37-fdc6c06333c2', 'f2ff1ade-050b-4639-b37f-69f1ec20f24b'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:24:03.820328 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:03.847470 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:03.874927 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:24:03.880146 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] Deleted volume successfully. Oct 01 21:24:03.892912 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d5bbbd13-1934-4f89-9079-9ff76530a837 tempest-VolumesNegativeTest-514173341 None] Lock "cinder-d4c67cfb-4b2e-4f8e-a16a-fcbc8b185503-delete_volume" released by "delete_volume" :: held 0.499s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:24:07.463389 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:24:07.482141 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (4f71ab52-6dfd-43a6-9219-f3afbad8701a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:24:07.488659 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9c535d03-d82f-43da-8fcb-ddf05b283df8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:07.504742 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9c535d03-d82f-43da-8fcb-ddf05b283df8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:24:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-479313879',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cda29501-6dd8-4f2b-ba75-8cb61dc80464,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='dd6e51791d934c73bf7d92e7ed1a8909',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:24:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:24:07Z,use_quota=True,user_id='98361240b014405cb20cf9c145061be6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:07.510545 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (636a050a-b4dd-4e75-8da8-f52324e9f636) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:07.514511 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (636a050a-b4dd-4e75-8da8-f52324e9f636) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:07.519379 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f7408670-641a-444a-b6de-87402bc9e369) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:07.523257 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f7408670-641a-444a-b6de-87402bc9e369) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'cda29501-6dd8-4f2b-ba75-8cb61dc80464', 'volume_name': 'volume-cda29501-6dd8-4f2b-ba75-8cb61dc80464', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:07.528016 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (52e10890-7c92-4fe0-baa5-10a217768c43) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:07.531828 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (52e10890-7c92-4fe0-baa5-10a217768c43) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:07.536275 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (331183d2-09a8-427a-82de-dc12205d224e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:07.537324 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Volume cda29501-6dd8-4f2b-ba75-8cb61dc80464: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-cda29501-6dd8-4f2b-ba75-8cb61dc80464', 'volume_size': 1} Oct 01 21:24:07.537722 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] creating volume 'volume-cda29501-6dd8-4f2b-ba75-8cb61dc80464' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:24:07.538527 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:07.596986 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:24:07.597365 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:24:07.600922 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (331183d2-09a8-427a-82de-dc12205d224e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cda29501-6dd8-4f2b-ba75-8cb61dc80464', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:07.605908 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (949d527e-e67b-4d95-9e87-2915e0a593aa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:07.615640 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Volume volume-cda29501-6dd8-4f2b-ba75-8cb61dc80464 (cda29501-6dd8-4f2b-ba75-8cb61dc80464): created successfully Oct 01 21:24:07.619717 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (949d527e-e67b-4d95-9e87-2915e0a593aa) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:07.624886 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (4f71ab52-6dfd-43a6-9219-f3afbad8701a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:24:07.633117 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-39c09684-26ad-416e-884e-80e295ec8c9c tempest-VolumesSnapshotTestJSON-1103874744 None] Created volume successfully. Oct 01 21:24:08.661871 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b5a68d6d-5cfd-438b-bcac-5a680df21456 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:09.271579 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=27,cinder:UPDATE=15,cinder:DELETE=2,cinder:INSERT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:09.289505 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b5a68d6d-5cfd-438b-bcac-5a680df21456 tempest-VolumesSnapshotTestJSON-1103874744 None] Create snapshot completed successfully Oct 01 21:24:09.907389 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.driver [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] Creating a new backup for volume volume-cda29501-6dd8-4f2b-ba75-8cb61dc80464. {{(pid=133391) _get_backup_volume_temp_volume /opt/stack/cinder/cinder/volume/driver.py:1218}} Oct 01 21:24:09.938961 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] cloning volumes/volume-cda29501-6dd8-4f2b-ba75-8cb61dc80464@snapshot-24b3dee7-ae24-428a-8d45-d1e94d233a60 to volume-e8aa1b25-dd5f-4fad-8217-5dd8db6e7d25 {{(pid=133391) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Oct 01 21:24:09.939608 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:09.975235 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:10.064623 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:24:10.065000 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:24:10.085404 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-10-01T21:24:09Z,data_timestamp=2022-10-01T21:24:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Backup-429649506',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=ea9e7a8a-a960-4cc0-bf0d-000007402b81,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='dd6e51791d934c73bf7d92e7ed1a8909',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id='24b3dee7-ae24-428a-8d45-d1e94d233a60',status='creating',temp_snapshot_id=None,temp_volume_id='e8aa1b25-dd5f-4fad-8217-5dd8db6e7d25',updated_at=None,user_id='98361240b014405cb20cf9c145061be6',volume_id=cda29501-6dd8-4f2b-ba75-8cb61dc80464) Oct 01 21:24:10.086156 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.backup.rpcapi [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] continue_backup in rpcapi backup_id ea9e7a8a-a960-4cc0-bf0d-000007402b81 {{(pid=133391) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Oct 01 21:24:10.178832 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:24:10.961267 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.782s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:24:10.963099 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e8aa1b25-dd5f-4fad-8217-5dd8db6e7d25', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'e8aa1b25-dd5f-4fad-8217-5dd8db6e7d25', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:24:10.980091 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] Initialize volume connection completed successfully. Oct 01 21:24:18.843509 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] Terminate volume connection completed successfully. Oct 01 21:24:18.864634 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] Remove volume export completed successfully. Oct 01 21:24:18.906791 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-e8aa1b25-dd5f-4fad-8217-5dd8db6e7d25-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:24:18.918977 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-e8aa1b25-dd5f-4fad-8217-5dd8db6e7d25-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:24:18.936964 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:18.993010 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:24:18.998755 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] deleting rbd volume volume-e8aa1b25-dd5f-4fad-8217-5dd8db6e7d25 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:24:19.226392 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] Deleted volume successfully. Oct 01 21:24:19.232142 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c86628a1-b844-461e-8c23-9787ac406921 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-e8aa1b25-dd5f-4fad-8217-5dd8db6e7d25-delete_volume" released by "delete_volume" :: held 0.313s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:24:23.817933 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:24:23.818776 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:23.843813 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:23.873676 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:24:31.298779 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:24:31.322995 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (0051a4bb-14fd-46d1-9b75-929e36813e53) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:24:31.336191 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d13832e6-f5e5-43a5-a3ce-b9b44cf5c068) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:31.358801 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d13832e6-f5e5-43a5-a3ce-b9b44cf5c068) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:24:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1242337224',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=7dea6e35-1c10-40b3-8361-2f40a38aaad1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='dd6e51791d934c73bf7d92e7ed1a8909',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:24:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:24:31Z,use_quota=True,user_id='98361240b014405cb20cf9c145061be6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:31.366232 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eb95647d-0d45-4d49-b02a-78849de115bc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:31.370347 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eb95647d-0d45-4d49-b02a-78849de115bc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:31.375302 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a94be580-67ad-4517-b50a-c6413c18b222) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:31.476539 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a94be580-67ad-4517-b50a-c6413c18b222) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '7dea6e35-1c10-40b3-8361-2f40a38aaad1', 'volume_name': 'volume-7dea6e35-1c10-40b3-8361-2f40a38aaad1', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:31.487670 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3c6c508e-dc9e-438c-b0cb-b6f342b83acb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:31.497286 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3c6c508e-dc9e-438c-b0cb-b6f342b83acb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:31.504432 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (44666ce9-633c-4570-a551-061619e10215) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:31.506731 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Volume 7dea6e35-1c10-40b3-8361-2f40a38aaad1: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-7dea6e35-1c10-40b3-8361-2f40a38aaad1', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 21:24:31.507064 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Cloning 7dea6e35-1c10-40b3-8361-2f40a38aaad1 from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 21:24:31.517980 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:24:31.533661 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.026s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:24:31.546252 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 24, 31, 539572)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:24:31.546724 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:24:31.547174 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 21:24:31.557985 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.024s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:24:31.558975 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 21:24:31.565653 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 21, 24, 31, 563012)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 21:24:31.566390 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 21:24:31.567104 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 21:24:31.581463 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:31.667976 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] creating snapshot='volume-7dea6e35-1c10-40b3-8361-2f40a38aaad1.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 21:24:32.432218 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-7dea6e35-1c10-40b3-8361-2f40a38aaad1.clone_snap' to 'volume-7dea6e35-1c10-40b3-8361-2f40a38aaad1' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 21:24:32.552552 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:24:32.553847 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:24:32.554384 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:24:32.571166 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 21:24:32.571885 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Marking volume 7dea6e35-1c10-40b3-8361-2f40a38aaad1 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:24:32.579865 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to 7dea6e35-1c10-40b3-8361-2f40a38aaad1. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 21:24:32.580287 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Creating volume glance metadata for volume 7dea6e35-1c10-40b3-8361-2f40a38aaad1 backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 21:24:32.634023 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (44666ce9-633c-4570-a551-061619e10215) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7dea6e35-1c10-40b3-8361-2f40a38aaad1', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:32.639176 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ff01117c-626c-44a1-b7ba-bdd97bef7718) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:24:32.648371 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Volume volume-7dea6e35-1c10-40b3-8361-2f40a38aaad1 (7dea6e35-1c10-40b3-8361-2f40a38aaad1): created successfully Oct 01 21:24:32.652114 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ff01117c-626c-44a1-b7ba-bdd97bef7718) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:24:32.657447 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (0051a4bb-14fd-46d1-9b75-929e36813e53) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:24:32.666165 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d3aab8c6-ec7a-4c47-9489-a38fd0974c8d tempest-VolumesSnapshotTestJSON-1103874744 None] Created volume successfully. Oct 01 21:24:33.898155 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=47,cinder:INSERT=12,cinder:UPDATE=18,cinder:DELETE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:33.901971 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-39dedaec-caad-41a9-a29c-cb61e187ce0c req-7bb64bde-7da1-4aea-a43b-476ad3c5c44c tempest-VolumesSnapshotTestJSON-1103874744 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:24:34.807643 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-39dedaec-caad-41a9-a29c-cb61e187ce0c req-7bb64bde-7da1-4aea-a43b-476ad3c5c44c tempest-VolumesSnapshotTestJSON-1103874744 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.905s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:24:34.808512 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-39dedaec-caad-41a9-a29c-cb61e187ce0c req-7bb64bde-7da1-4aea-a43b-476ad3c5c44c tempest-VolumesSnapshotTestJSON-1103874744 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-7dea6e35-1c10-40b3-8361-2f40a38aaad1', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '7dea6e35-1c10-40b3-8361-2f40a38aaad1', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:24:34.838347 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-39dedaec-caad-41a9-a29c-cb61e187ce0c req-7bb64bde-7da1-4aea-a43b-476ad3c5c44c tempest-VolumesSnapshotTestJSON-1103874744 None] Connection info returned from driver {'name': 'volumes/volume-7dea6e35-1c10-40b3-8361-2f40a38aaad1', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '7dea6e35-1c10-40b3-8361-2f40a38aaad1', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'd56ed59a-8d47-4bfa-8460-261aa9e92b63'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:24:34.908176 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-39dedaec-caad-41a9-a29c-cb61e187ce0c req-7bb64bde-7da1-4aea-a43b-476ad3c5c44c tempest-VolumesSnapshotTestJSON-1103874744 None] attachment_update completed successfully. Oct 01 21:24:37.310289 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d5be7119-9083-4b7c-ac37-2468c4e431f8 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:37.600425 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d5be7119-9083-4b7c-ac37-2468c4e431f8 tempest-VolumesSnapshotTestJSON-1103874744 None] Create snapshot completed successfully Oct 01 21:24:38.636931 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1d1f0496-be58-46ec-9a9c-6ea2ecb8dbce tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:39.564399 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=57,cinder:UPDATE=11,cinder:INSERT=10,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:24:39.621228 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1d1f0496-be58-46ec-9a9c-6ea2ecb8dbce tempest-VolumesSnapshotTestJSON-1103874744 None] Create snapshot completed successfully Oct 01 21:24:40.905880 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-2207f134-2296-430f-899d-165db8a76193 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:41.662295 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-2207f134-2296-430f-899d-165db8a76193 tempest-VolumesSnapshotTestJSON-1103874744 None] Create snapshot completed successfully Oct 01 21:24:41.958248 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-cec743eb-3ee2-41df-9236-b0a794f35c3b tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-e6c2c32d-f08e-43d2-a27b-41f90446f377-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:24:41.968672 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-cec743eb-3ee2-41df-9236-b0a794f35c3b tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-e6c2c32d-f08e-43d2-a27b-41f90446f377-delete_snapshot" acquired by "delete_snapshot" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:24:41.970144 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cec743eb-3ee2-41df-9236-b0a794f35c3b tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:24:55.231515 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=36,cinder:UPDATE=8,cinder:INSERT=25,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:05.234604 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:INSERT=6,cinder:SELECT=16,cinder:DELETE=1,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:12.117871 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-cec743eb-3ee2-41df-9236-b0a794f35c3b tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['f02ad15f-f7e8-489f-8ad5-435fe1c75fa9', '9495096b-0da0-4679-92e0-72f9c11ac77d', 'a487d2f5-3308-4a1b-94c6-1f6f9d1392f5', '9f669237-6f17-4db6-a705-0e2a30cabbaa'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:25:12.156470 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-cec743eb-3ee2-41df-9236-b0a794f35c3b tempest-VolumesSnapshotTestJSON-1103874744 None] Delete snapshot completed successfully. Oct 01 21:25:12.160969 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-cec743eb-3ee2-41df-9236-b0a794f35c3b tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-e6c2c32d-f08e-43d2-a27b-41f90446f377-delete_snapshot" released by "delete_snapshot" :: held 30.192s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:25:12.926993 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7368d4ac-8632-46c7-ad3a-1340b941032f tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-3584f5a7-94a2-4525-88e1-8359dee00099-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:25:12.936658 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7368d4ac-8632-46c7-ad3a-1340b941032f tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-3584f5a7-94a2-4525-88e1-8359dee00099-delete_snapshot" acquired by "delete_snapshot" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:25:12.938605 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7368d4ac-8632-46c7-ad3a-1340b941032f tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:25:23.818796 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:25:23.820020 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:25:23.845277 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:25:23.869449 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:25:25.236180 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=13,cinder:UPDATE=6,cinder:INSERT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:35.246887 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=14,cinder:INSERT=2,cinder:UPDATE=8 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:39.692538 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-1e22f5b2-4299-49ae-aa28-ce9364628876 req-881aa0d9-2300-46db-8643-938fdc6ee9f2 tempest-TestVolumeBootPattern-1545759599 None] Terminate volume connection completed successfully. Oct 01 21:25:39.693105 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-1e22f5b2-4299-49ae-aa28-ce9364628876 req-881aa0d9-2300-46db-8643-938fdc6ee9f2 tempest-TestVolumeBootPattern-1545759599 None] Deleting attachment c79719d8-75ae-45fa-b6ed-87600ec992f0. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:25:42.857585 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-fec99b6c-eeac-472e-981d-77b40f522485 req-e137dc7b-7137-4c65-ae1c-8622917d4546 tempest-TestVolumeBootPattern-1545759599 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:25:43.133254 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-7368d4ac-8632-46c7-ad3a-1340b941032f tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['be2880b8-a486-47ff-a8bb-98ea6c458669', '97b8428e-96dc-476f-bd09-aa04ad2acc68', '8f1bb45e-1f10-4ffe-86ee-4a2e3d4ae31a', '01ee4430-a0e6-40cc-a659-7e28cd589534'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:25:43.195505 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7368d4ac-8632-46c7-ad3a-1340b941032f tempest-VolumesSnapshotTestJSON-1103874744 None] Delete snapshot completed successfully. Oct 01 21:25:43.202331 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-7368d4ac-8632-46c7-ad3a-1340b941032f tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-3584f5a7-94a2-4525-88e1-8359dee00099-delete_snapshot" released by "delete_snapshot" :: held 30.266s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:25:43.813010 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-fec99b6c-eeac-472e-981d-77b40f522485 req-e137dc7b-7137-4c65-ae1c-8622917d4546 tempest-TestVolumeBootPattern-1545759599 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.955s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:25:43.813802 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-fec99b6c-eeac-472e-981d-77b40f522485 req-e137dc7b-7137-4c65-ae1c-8622917d4546 tempest-TestVolumeBootPattern-1545759599 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:25:43.849970 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-fec99b6c-eeac-472e-981d-77b40f522485 req-e137dc7b-7137-4c65-ae1c-8622917d4546 tempest-TestVolumeBootPattern-1545759599 None] Connection info returned from driver {'name': 'volumes/volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '91a33031-458c-4c6f-8fe0-fccc855ffd7f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'c8780623-6e28-49b2-811b-93c327b0aae5'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:25:43.937496 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a44e8f13-2f89-48dc-9a61-6d72b13b27e5 tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-4d998fa7-e388-42de-b1ca-1d67134f637a-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:25:43.940943 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-fec99b6c-eeac-472e-981d-77b40f522485 req-e137dc7b-7137-4c65-ae1c-8622917d4546 tempest-TestVolumeBootPattern-1545759599 None] attachment_update completed successfully. Oct 01 21:25:43.958845 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a44e8f13-2f89-48dc-9a61-6d72b13b27e5 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-4d998fa7-e388-42de-b1ca-1d67134f637a-delete_snapshot" acquired by "delete_snapshot" :: waited 0.021s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:25:43.970271 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a44e8f13-2f89-48dc-9a61-6d72b13b27e5 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:25:53.913575 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=25,cinder:UPDATE=11,cinder:INSERT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:25:55.254536 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=24,cinder:INSERT=2,cinder:UPDATE=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:14.189602 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-a44e8f13-2f89-48dc-9a61-6d72b13b27e5 tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['45bb9009-f815-4c6e-8a33-2c21cc1b92dc', '298d8902-aa65-4029-bffd-649901c4ea64', '8a643291-5a63-46fd-b1bd-3ab82a10e135', 'a76100e3-4491-439f-a454-454032aec534'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:26:14.237505 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a44e8f13-2f89-48dc-9a61-6d72b13b27e5 tempest-VolumesSnapshotTestJSON-1103874744 None] Delete snapshot completed successfully. Oct 01 21:26:14.243438 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a44e8f13-2f89-48dc-9a61-6d72b13b27e5 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-4d998fa7-e388-42de-b1ca-1d67134f637a-delete_snapshot" released by "delete_snapshot" :: held 30.285s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:26:15.769033 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-5e481010-0de7-4abc-8dbf-3d7d05d70153 req-573a2499-61ea-4b45-884a-55224b911757 tempest-VolumesSnapshotTestJSON-1103874744 None] Terminate volume connection completed successfully. Oct 01 21:26:15.770140 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-5e481010-0de7-4abc-8dbf-3d7d05d70153 req-573a2499-61ea-4b45-884a-55224b911757 tempest-VolumesSnapshotTestJSON-1103874744 None] Deleting attachment d56ed59a-8d47-4bfa-8460-261aa9e92b63. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:26:20.307475 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8f4b9715-4cf1-4705-bf65-e155dc31ce7d tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:26:21.352894 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8f4b9715-4cf1-4705-bf65-e155dc31ce7d tempest-VolumesSnapshotTestJSON-1103874744 None] Create snapshot completed successfully Oct 01 21:26:22.530892 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d76d8078-dc57-4e86-894d-ade84d69c4d2 tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-83797689-088e-4985-9454-dae0804d0d5b-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:26:22.541777 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d76d8078-dc57-4e86-894d-ade84d69c4d2 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-83797689-088e-4985-9454-dae0804d0d5b-delete_snapshot" acquired by "delete_snapshot" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:26:22.543954 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d76d8078-dc57-4e86-894d-ade84d69c4d2 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:26:23.382276 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-d76d8078-dc57-4e86-894d-ade84d69c4d2 tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['63c765c8-9e5b-4ab5-aa14-f81dd2317a5a', '99cbe48d-c2be-4c88-865a-b38959f3cc00', 'd515d73b-25be-4597-9661-bb95d82e876b', 'e36b9b12-7424-4321-b893-decf07ac17ff'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:26:23.419272 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d76d8078-dc57-4e86-894d-ade84d69c4d2 tempest-VolumesSnapshotTestJSON-1103874744 None] Delete snapshot completed successfully. Oct 01 21:26:23.428355 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d76d8078-dc57-4e86-894d-ade84d69c4d2 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-83797689-088e-4985-9454-dae0804d0d5b-delete_snapshot" released by "delete_snapshot" :: held 0.887s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:26:23.798068 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0d12c3cd-5444-4723-b10b-867a4afb7445 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:26:23.838605 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:26:23.839427 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:26:23.867286 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:26:23.901368 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:26:24.374831 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=34,cinder:UPDATE=14,cinder:INSERT=3,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:24.380615 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0d12c3cd-5444-4723-b10b-867a4afb7445 tempest-VolumesSnapshotTestJSON-1103874744 None] Create snapshot completed successfully Oct 01 21:26:25.266896 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=34,cinder:UPDATE=15,cinder:INSERT=5,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:26:35.881520 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-08428f75-77f3-4483-9ef8-fab8304fd8b8 req-e4a09574-e538-44ae-9864-0aa656247cbf tempest-VolumesSnapshotTestJSON-1103874744 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:26:36.698246 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-08428f75-77f3-4483-9ef8-fab8304fd8b8 req-e4a09574-e538-44ae-9864-0aa656247cbf tempest-VolumesSnapshotTestJSON-1103874744 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.816s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:26:36.699647 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-08428f75-77f3-4483-9ef8-fab8304fd8b8 req-e4a09574-e538-44ae-9864-0aa656247cbf tempest-VolumesSnapshotTestJSON-1103874744 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-cda29501-6dd8-4f2b-ba75-8cb61dc80464', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'cda29501-6dd8-4f2b-ba75-8cb61dc80464', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:26:36.728545 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-08428f75-77f3-4483-9ef8-fab8304fd8b8 req-e4a09574-e538-44ae-9864-0aa656247cbf tempest-VolumesSnapshotTestJSON-1103874744 None] Connection info returned from driver {'name': 'volumes/volume-cda29501-6dd8-4f2b-ba75-8cb61dc80464', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'cda29501-6dd8-4f2b-ba75-8cb61dc80464', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'c4595c42-725d-488a-8404-28d1e5cf986c'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:26:36.788653 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-08428f75-77f3-4483-9ef8-fab8304fd8b8 req-e4a09574-e538-44ae-9864-0aa656247cbf tempest-VolumesSnapshotTestJSON-1103874744 None] attachment_update completed successfully. Oct 01 21:26:39.270062 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-02ecb642-12a0-43d9-bcc9-5a06bdaca10f tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:26:39.512440 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-02ecb642-12a0-43d9-bcc9-5a06bdaca10f tempest-VolumesSnapshotTestJSON-1103874744 None] Create snapshot completed successfully Oct 01 21:26:40.546209 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-46305bfd-d9ff-48e3-83d8-40d770c8b77a tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:26:41.515580 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-46305bfd-d9ff-48e3-83d8-40d770c8b77a tempest-VolumesSnapshotTestJSON-1103874744 None] Create snapshot completed successfully Oct 01 21:26:41.610848 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f8750baf-0a91-4b98-a256-bed728def67a tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-50283759-8b18-4613-83ff-b13c690ae0e1-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:26:41.624310 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f8750baf-0a91-4b98-a256-bed728def67a tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-50283759-8b18-4613-83ff-b13c690ae0e1-delete_snapshot" acquired by "delete_snapshot" :: waited 0.013s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:26:41.626272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f8750baf-0a91-4b98-a256-bed728def67a tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:26:55.262973 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=30,cinder:UPDATE=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:05.268848 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=25,cinder:UPDATE=6,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:11.783274 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-f8750baf-0a91-4b98-a256-bed728def67a tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['2f6774c8-de8d-48e5-9cb7-0b190c0ceb24', 'f036836a-b099-41d8-90f0-6170e7057b20', '907670e7-2a36-450f-a660-d89b096cfc26', '485978f6-33cc-42f4-a6e6-972124922476'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:27:11.831441 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f8750baf-0a91-4b98-a256-bed728def67a tempest-VolumesSnapshotTestJSON-1103874744 None] Delete snapshot completed successfully. Oct 01 21:27:11.843710 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f8750baf-0a91-4b98-a256-bed728def67a tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-50283759-8b18-4613-83ff-b13c690ae0e1-delete_snapshot" released by "delete_snapshot" :: held 30.220s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:27:12.688958 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d7dbc173-3739-4e80-b257-f55354d7b12f tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-998b66c7-6e1b-4eab-9cf9-0ff4601f8bb5-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:27:12.699627 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d7dbc173-3739-4e80-b257-f55354d7b12f tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-998b66c7-6e1b-4eab-9cf9-0ff4601f8bb5-delete_snapshot" acquired by "delete_snapshot" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:27:12.701509 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d7dbc173-3739-4e80-b257-f55354d7b12f tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:23.837442 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:27:23.838314 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:23.863858 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:23.888082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:27:25.267603 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=12,cinder:UPDATE=7,cinder:INSERT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:29.852017 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0b903ec3-3a7d-4867-b183-f12e6893a3c3 tempest-BackendsCapabilitiesAdminTestsJSON-1372850760 None] Obtained capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.3.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:3f02706e-6a89-4cc7-a5d0-346f6e188c25:cinder:volumes', 'backend_state': 'up', 'qos_support': True, 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=133391) get_capabilities /opt/stack/cinder/cinder/volume/manager.py:4776}} Oct 01 21:27:29.885499 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e79e00cc-1dd9-4b5a-ad22-614df0b82753 tempest-BackendsCapabilitiesAdminTestsJSON-1372850760 None] Obtained capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.3.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:3f02706e-6a89-4cc7-a5d0-346f6e188c25:cinder:volumes', 'backend_state': 'up', 'qos_support': True, 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=133391) get_capabilities /opt/stack/cinder/cinder/volume/manager.py:4776}} Oct 01 21:27:32.623457 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f4bb3c5a-cf93-483b-b900-20a9c3b9b5ab tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:32.732252 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-79246af3-7e79-4d54-b29d-744142805a7c tempest-GroupSnapshotsV319Test-428881024 None] Group tempest-GroupSnapshotsV319Test-Group-1929074343: creating Oct 01 21:27:32.749471 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-79246af3-7e79-4d54-b29d-744142805a7c tempest-GroupSnapshotsV319Test-428881024 None] Group tempest-GroupSnapshotsV319Test-Group-1929074343: created successfully Oct 01 21:27:32.758573 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-79246af3-7e79-4d54-b29d-744142805a7c tempest-GroupSnapshotsV319Test-428881024 None] Create group completed successfully. Oct 01 21:27:33.541637 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f4bb3c5a-cf93-483b-b900-20a9c3b9b5ab tempest-TestVolumeBootPattern-1545759599 None] Create snapshot completed successfully Oct 01 21:27:34.113706 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:27:34.145900 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (4f3f057f-bf0d-4120-82a3-d1021cb72b65) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:27:34.152677 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7f99047c-480e-472d-a2a0-aed219589f18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.178819 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7f99047c-480e-472d-a2a0-aed219589f18) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:27:34Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-1898884961',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a61c69d6-7d79-413d-87ad-e254e121f623,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='df0e56b020444dd28af868590b1bccf4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:27:34Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=56b61d13-1043-4d8c-b993-cc2ef8a3f3ec,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:27:34Z,use_quota=True,user_id='eb0bf5fce885439eaa16a0871c2992ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.184018 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4630e2f9-5067-4365-a050-3a32cfef995b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.188521 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4630e2f9-5067-4365-a050-3a32cfef995b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.193493 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (51c07297-1f64-45f9-bec3-b2d50ff198f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.197522 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (51c07297-1f64-45f9-bec3-b2d50ff198f9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'a61c69d6-7d79-413d-87ad-e254e121f623', 'volume_name': 'volume-a61c69d6-7d79-413d-87ad-e254e121f623', 'volume_size': 1, 'snapshot_id': '56b61d13-1043-4d8c-b993-cc2ef8a3f3ec'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.202851 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fd1fa77a-7052-4e7a-9351-a38d4becdd1d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.206727 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fd1fa77a-7052-4e7a-9351-a38d4becdd1d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.211273 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (558de32f-a9ea-455a-bd69-a0f7264c3a0f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.212302 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Volume a61c69d6-7d79-413d-87ad-e254e121f623: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-a61c69d6-7d79-413d-87ad-e254e121f623', 'volume_size': 1, 'snapshot_id': '56b61d13-1043-4d8c-b993-cc2ef8a3f3ec'} Oct 01 21:27:34.227990 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:27:34.246699 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Flow 'volume_create_manager' (85c2b257-4bbc-4309-86a7-a2944a9387bc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:27:34.253485 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (940a1125-85bf-4c2a-8e75-1d1883514c87) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.274116 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (940a1125-85bf-4c2a-8e75-1d1883514c87) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:27:34Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsV319Test-Volume-1823376098',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=09f14abe-6ab8-44ea-a783-e24fafd75d93,host='ncdv-master-1@ceph#ceph',id=b8291b05-49c3-4d36-9d8e-9f17a5bc5239,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ef8c8e969dc040d89f85e78613bf984c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:27:34Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:27:34Z,use_quota=True,user_id='ae06c9808a3f4aecb041b36d505ee9b4',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(2aac6aee-2865-494f-855d-249fd876c5e4),volume_type_id=2aac6aee-2865-494f-855d-249fd876c5e4)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.279285 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3f22fbdd-36bb-47d4-ace0-2b74ee9e6c64) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.283282 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3f22fbdd-36bb-47d4-ace0-2b74ee9e6c64) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.288178 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7968ee61-eaf4-481a-a0d3-a32ac1779802) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.292093 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7968ee61-eaf4-481a-a0d3-a32ac1779802) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b8291b05-49c3-4d36-9d8e-9f17a5bc5239', 'volume_name': 'volume-b8291b05-49c3-4d36-9d8e-9f17a5bc5239', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.297598 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f7d5056f-fa69-4ce4-b7a0-868355bb246b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.301352 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f7d5056f-fa69-4ce4-b7a0-868355bb246b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.305890 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b5453452-97e1-46ff-84b6-7d9882c9bca6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.306999 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Volume b8291b05-49c3-4d36-9d8e-9f17a5bc5239: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b8291b05-49c3-4d36-9d8e-9f17a5bc5239', 'volume_size': 1} Oct 01 21:27:34.307395 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] creating volume 'volume-b8291b05-49c3-4d36-9d8e-9f17a5bc5239' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:27:34.308147 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:34.335616 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] cloning volumes/volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f@snapshot-56b61d13-1043-4d8c-b993-cc2ef8a3f3ec to volume-a61c69d6-7d79-413d-87ad-e254e121f623 {{(pid=133391) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Oct 01 21:27:34.336250 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:34.393597 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:34.429515 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:27:34.429979 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:27:34.433816 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b5453452-97e1-46ff-84b6-7d9882c9bca6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b8291b05-49c3-4d36-9d8e-9f17a5bc5239', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.439083 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ac2f3802-9688-4fa0-b68a-e8f3ebb0d217) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.448394 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Volume volume-b8291b05-49c3-4d36-9d8e-9f17a5bc5239 (b8291b05-49c3-4d36-9d8e-9f17a5bc5239): created successfully Oct 01 21:27:34.451976 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ac2f3802-9688-4fa0-b68a-e8f3ebb0d217) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.457415 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Flow 'volume_create_manager' (85c2b257-4bbc-4309-86a7-a2944a9387bc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:27:34.466598 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-93fa521c-8189-4041-9728-eda095ee4c7d tempest-GroupSnapshotsV319Test-428881024 None] Created volume successfully. Oct 01 21:27:34.524676 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:27:34.525056 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:27:34.544982 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Marking volume a61c69d6-7d79-413d-87ad-e254e121f623 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 21:27:34.551979 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Copying metadata from snapshot 56b61d13-1043-4d8c-b993-cc2ef8a3f3ec to a61c69d6-7d79-413d-87ad-e254e121f623. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:431}} Oct 01 21:27:34.586082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (558de32f-a9ea-455a-bd69-a0f7264c3a0f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a61c69d6-7d79-413d-87ad-e254e121f623', 'volume_size': 1, 'snapshot_id': '56b61d13-1043-4d8c-b993-cc2ef8a3f3ec'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.590461 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (05367733-84c6-4a8b-aefa-1acc0d2d60a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:34.598914 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Volume volume-a61c69d6-7d79-413d-87ad-e254e121f623 (a61c69d6-7d79-413d-87ad-e254e121f623): created successfully Oct 01 21:27:34.602202 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (05367733-84c6-4a8b-aefa-1acc0d2d60a3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:34.608092 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Flow 'volume_create_manager' (4f3f057f-bf0d-4120-82a3-d1021cb72b65) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:27:34.619545 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-8d3fa46d-760e-4e73-b99e-83a63466ec8b tempest-TestVolumeBootPattern-1545759599 None] Created volume successfully. Oct 01 21:27:35.389156 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-09f962aa-d139-44f5-b166-988012eed587 tempest-GroupSnapshotsV319Test-428881024 None] GroupSnapshot f2c71d36-87c1-46f5-a82d-839966942717: creating. Oct 01 21:27:35.401774 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=41,cinder:INSERT=18,cinder:UPDATE=14,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:35.408731 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-09f962aa-d139-44f5-b166-988012eed587 tempest-GroupSnapshotsV319Test-428881024 None] Group snapshot f2c71d36-87c1-46f5-a82d-839966942717: creating. {{(pid=133391) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Oct 01 21:27:35.428299 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-09f962aa-d139-44f5-b166-988012eed587 tempest-GroupSnapshotsV319Test-428881024 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:36.577161 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-09f962aa-d139-44f5-b166-988012eed587 tempest-GroupSnapshotsV319Test-428881024 None] group_snapshot f2c71d36-87c1-46f5-a82d-839966942717: created successfully Oct 01 21:27:37.533124 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-e3d862e7-c25f-443b-bc48-e586196482bf req-00cb98f5-376c-4657-ba42-ab2de83277b3 tempest-TestVolumeBootPattern-1545759599 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:27:37.894934 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0382fa99-973e-42ef-8dba-4ecd14b96c1f tempest-GroupSnapshotsV319Test-428881024 None] group_snapshot f2c71d36-87c1-46f5-a82d-839966942717: deleting Oct 01 21:27:37.916827 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-0382fa99-973e-42ef-8dba-4ecd14b96c1f tempest-GroupSnapshotsV319Test-428881024 None] group_snapshot f2c71d36-87c1-46f5-a82d-839966942717: deleting {{(pid=133391) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Oct 01 21:27:37.937258 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-0382fa99-973e-42ef-8dba-4ecd14b96c1f tempest-GroupSnapshotsV319Test-428881024 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:38.379258 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-e3d862e7-c25f-443b-bc48-e586196482bf req-00cb98f5-376c-4657-ba42-ab2de83277b3 tempest-TestVolumeBootPattern-1545759599 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.846s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:27:38.380076 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-e3d862e7-c25f-443b-bc48-e586196482bf req-00cb98f5-376c-4657-ba42-ab2de83277b3 tempest-TestVolumeBootPattern-1545759599 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-a61c69d6-7d79-413d-87ad-e254e121f623', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': 'a61c69d6-7d79-413d-87ad-e254e121f623', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:27:38.401938 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-e3d862e7-c25f-443b-bc48-e586196482bf req-00cb98f5-376c-4657-ba42-ab2de83277b3 tempest-TestVolumeBootPattern-1545759599 None] Connection info returned from driver {'name': 'volumes/volume-a61c69d6-7d79-413d-87ad-e254e121f623', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'a61c69d6-7d79-413d-87ad-e254e121f623', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '285c8113-fde3-465b-82bd-c6fe44ac0492'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:27:38.451913 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-e3d862e7-c25f-443b-bc48-e586196482bf req-00cb98f5-376c-4657-ba42-ab2de83277b3 tempest-TestVolumeBootPattern-1545759599 None] attachment_update completed successfully. Oct 01 21:27:38.604146 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-0382fa99-973e-42ef-8dba-4ecd14b96c1f tempest-GroupSnapshotsV319Test-428881024 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsV319Test-volume-type-77329806 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsV319Test-volume-type-77329806, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:27:38.605565 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-0382fa99-973e-42ef-8dba-4ecd14b96c1f tempest-GroupSnapshotsV319Test-428881024 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsV319Test-volume-type-77329806 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsV319Test-volume-type-77329806, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:27:38.626322 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-0382fa99-973e-42ef-8dba-4ecd14b96c1f tempest-GroupSnapshotsV319Test-428881024 None] Created reservations ['fdb778e5-15b6-4c41-8a5c-cbeab4934ab1', '37a6f810-34a4-4d5e-a6a4-e2f26ef2904a', 'c302017e-7266-474b-9bec-e750fc4105c3', 'd122dec3-1db8-433b-9b00-674d818b62d8'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:27:38.659563 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=79,cinder:UPDATE=8,cinder:INSERT=11,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:38.699684 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-0382fa99-973e-42ef-8dba-4ecd14b96c1f tempest-GroupSnapshotsV319Test-428881024 None] group_snapshot f2c71d36-87c1-46f5-a82d-839966942717: deleted successfully Oct 01 21:27:39.342046 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cd1f6c18-e6dd-43e4-b620-f8595ca40eac tempest-GroupSnapshotsV319Test-428881024 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:39.380734 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cd1f6c18-e6dd-43e4-b620-f8595ca40eac tempest-GroupSnapshotsV319Test-428881024 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:27:39.381248 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cd1f6c18-e6dd-43e4-b620-f8595ca40eac tempest-GroupSnapshotsV319Test-428881024 None] Volume volume-b8291b05-49c3-4d36-9d8e-9f17a5bc5239 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:27:39.385227 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cd1f6c18-e6dd-43e4-b620-f8595ca40eac tempest-GroupSnapshotsV319Test-428881024 None] deleting rbd volume volume-b8291b05-49c3-4d36-9d8e-9f17a5bc5239 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:27:39.563354 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-cd1f6c18-e6dd-43e4-b620-f8595ca40eac tempest-GroupSnapshotsV319Test-428881024 None] Created reservations ['98dc5391-4466-43e9-bc71-356e35b1910c'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:27:39.583245 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-cd1f6c18-e6dd-43e4-b620-f8595ca40eac tempest-GroupSnapshotsV319Test-428881024 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsV319Test-volume-type-77329806 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsV319Test-volume-type-77329806, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:27:39.597837 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-cd1f6c18-e6dd-43e4-b620-f8595ca40eac tempest-GroupSnapshotsV319Test-428881024 None] Created reservations ['f4b5a4e3-a032-4e22-b2e8-bd99240295fe', '21aa2de9-3d45-48db-8400-501f94408d04', 'd6357473-bc8b-4f56-93d8-99bb36090719', '3ac0afdc-38f4-4e6e-b688-767e70eccb0a'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:27:39.676914 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cd1f6c18-e6dd-43e4-b620-f8595ca40eac tempest-GroupSnapshotsV319Test-428881024 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:39.703158 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-cd1f6c18-e6dd-43e4-b620-f8595ca40eac tempest-GroupSnapshotsV319Test-428881024 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:39.738274 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-cd1f6c18-e6dd-43e4-b620-f8595ca40eac tempest-GroupSnapshotsV319Test-428881024 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:27:39.748008 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-cd1f6c18-e6dd-43e4-b620-f8595ca40eac tempest-GroupSnapshotsV319Test-428881024 None] Delete group completed successfully. Oct 01 21:27:42.850700 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-d7dbc173-3739-4e80-b257-f55354d7b12f tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['a46c73c7-bd59-41e4-83a2-405dd071cac7', '54246c7d-1142-40f6-a1d4-b9363a1fb382', '06ceb865-23c5-43b7-b285-43482c081b16', 'f4abf2ed-c824-4aaf-87e0-be1af7def0ad'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:27:42.892212 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d7dbc173-3739-4e80-b257-f55354d7b12f tempest-VolumesSnapshotTestJSON-1103874744 None] Delete snapshot completed successfully. Oct 01 21:27:42.898903 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d7dbc173-3739-4e80-b257-f55354d7b12f tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-998b66c7-6e1b-4eab-9cf9-0ff4601f8bb5-delete_snapshot" released by "delete_snapshot" :: held 30.200s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:27:43.672511 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:27:43.692823 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Flow 'volume_create_manager' (46dcd6d0-ade4-46ec-ac6b-98239178b8d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:27:43.699224 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cbe5bfbc-0449-459a-aa29-708d8a2218e7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:43.715360 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cbe5bfbc-0449-459a-aa29-708d8a2218e7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:27:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeRetypeWithoutMigrationTest-Volume-182037636',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=59259c40-46e7-4900-aab4-facafbaa8212,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a92472a49e5c4940882f847a9b000bae',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:27:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:27:44Z,use_quota=True,user_id='f16e8200d63340188821b6d9ed34577a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(db88b327-33f6-459f-9eab-9d1e3e577e7b),volume_type_id=db88b327-33f6-459f-9eab-9d1e3e577e7b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:43.720565 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b907440f-abdb-4fcb-b0d4-fc29b5b86974) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:43.724570 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b907440f-abdb-4fcb-b0d4-fc29b5b86974) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:43.729550 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a58aa2ad-c639-44a7-a95e-d36aea7db723) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:43.733703 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a58aa2ad-c639-44a7-a95e-d36aea7db723) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '59259c40-46e7-4900-aab4-facafbaa8212', 'volume_name': 'volume-59259c40-46e7-4900-aab4-facafbaa8212', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:43.738463 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (13e001d6-f784-4852-90bc-88a893f6eb67) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:43.742275 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (13e001d6-f784-4852-90bc-88a893f6eb67) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:43.746885 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (07e954fb-6bca-4af7-889c-39407e7071d3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:43.748471 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Volume 59259c40-46e7-4900-aab4-facafbaa8212: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-59259c40-46e7-4900-aab4-facafbaa8212', 'volume_size': 1} Oct 01 21:27:43.748869 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] creating volume 'volume-59259c40-46e7-4900-aab4-facafbaa8212' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:27:43.749860 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:43.797064 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] extra_specs: {} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:27:43.797503 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] extra_specs: {} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:27:43.801112 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (07e954fb-6bca-4af7-889c-39407e7071d3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-59259c40-46e7-4900-aab4-facafbaa8212', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:43.805576 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e02f8fb5-92e2-40ef-81a9-d796b92579b0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:43.815507 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Volume volume-59259c40-46e7-4900-aab4-facafbaa8212 (59259c40-46e7-4900-aab4-facafbaa8212): created successfully Oct 01 21:27:43.819065 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e02f8fb5-92e2-40ef-81a9-d796b92579b0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:43.824674 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Flow 'volume_create_manager' (46dcd6d0-ade4-46ec-ac6b-98239178b8d7) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:27:43.834884 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-7f4ccd41-fccd-49e6-b536-1adbad507a41 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Created volume successfully. Oct 01 21:27:43.887173 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1878f4aa-98d0-41f4-aa20-37ee1e10682b tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-52f8aea8-115f-4d52-8466-cead4319e408-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:27:43.896059 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1878f4aa-98d0-41f4-aa20-37ee1e10682b tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-52f8aea8-115f-4d52-8466-cead4319e408-delete_snapshot" acquired by "delete_snapshot" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:27:43.897546 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1878f4aa-98d0-41f4-aa20-37ee1e10682b tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:44.944696 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=37,cinder:SELECT=57,cinder:INSERT=5,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:45.015162 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d070d270-212a-4b71-93cc-773b4d34d610 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:45.039476 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d070d270-212a-4b71-93cc-773b4d34d610 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:45.063623 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-d070d270-212a-4b71-93cc-773b4d34d610 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:27:45.067916 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d070d270-212a-4b71-93cc-773b4d34d610 tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Retype volume completed successfully. Oct 01 21:27:46.213955 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Acquiring lock "cinder-59259c40-46e7-4900-aab4-facafbaa8212-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:27:46.223201 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Lock "cinder-59259c40-46e7-4900-aab4-facafbaa8212-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:27:46.261304 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:46.303799 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:27:46.304225 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Volume volume-59259c40-46e7-4900-aab4-facafbaa8212 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:27:46.309530 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] deleting rbd volume volume-59259c40-46e7-4900-aab4-facafbaa8212 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:27:46.470197 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Deprecated: Default quota for resource: volumes_volume-type-2 is set by the default quota flag: quota_volumes_volume-type-2, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:27:46.470973 ncdv-master-1 cinder-volume[133391]: WARNING cinder.quota [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Deprecated: Default quota for resource: gigabytes_volume-type-2 is set by the default quota flag: quota_gigabytes_volume-type-2, it is now deprecated. Please use the default quota class for default quota. Oct 01 21:27:46.485333 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Created reservations ['6c80f054-3ee9-4e9d-9ae6-cebc82bc4a83', 'a7c011da-c177-4c8a-9d2e-99e9d9a92c64', '5181c8f3-c4da-47dd-b64b-c2928faa242d', 'd62f6e1e-ae4a-481d-bf6c-04673f16cad9'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:27:46.529302 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:46.560626 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:46.596499 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:27:46.600597 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Deleted volume successfully. Oct 01 21:27:46.614466 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-decf1de7-9e1a-4f75-9295-544d1ebd363b tempest-VolumeRetypeWithoutMigrationTest-410188473 None] Lock "cinder-59259c40-46e7-4900-aab4-facafbaa8212-delete_volume" released by "delete_volume" :: held 0.391s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:27:50.038435 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:27:50.057402 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Flow 'volume_create_manager' (5bcce63c-1f53-424a-9569-29f8482971ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:27:50.063687 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ac0d3c13-eacf-4da6-8525-71a7861787e0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:50.079989 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ac0d3c13-eacf-4da6-8525-71a7861787e0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:27:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotNegativeTestJSON-Volume-1637072780',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6110d45f-af58-408f-88e7-bd46665d58c3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='291dbdcbbe824d0ea41ba7f5152148b9',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:27:50Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:27:50Z,use_quota=True,user_id='19579ef261d7403c80f415c0ca12f8ed',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:50.085595 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (334bd89f-780c-46e0-b605-9eaeda9cb95c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:50.089598 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (334bd89f-780c-46e0-b605-9eaeda9cb95c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:50.095124 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6a40764a-9ea6-4d95-8110-30befbea7dad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:50.099070 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6a40764a-9ea6-4d95-8110-30befbea7dad) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6110d45f-af58-408f-88e7-bd46665d58c3', 'volume_name': 'volume-6110d45f-af58-408f-88e7-bd46665d58c3', 'volume_size': 2}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:50.103813 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c5167fea-528b-4924-ab95-10f7b2befc8f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:50.107643 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c5167fea-528b-4924-ab95-10f7b2befc8f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:50.112181 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dd912008-7134-4919-974e-fd3b743da705) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:50.113238 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Volume 6110d45f-af58-408f-88e7-bd46665d58c3: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6110d45f-af58-408f-88e7-bd46665d58c3', 'volume_size': 2} Oct 01 21:27:50.113636 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] creating volume 'volume-6110d45f-af58-408f-88e7-bd46665d58c3' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:27:50.114409 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:50.160539 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:27:50.160923 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:27:50.164562 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dd912008-7134-4919-974e-fd3b743da705) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6110d45f-af58-408f-88e7-bd46665d58c3', 'volume_size': 2}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:50.169303 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (412c5e55-8978-4587-9030-7a80595cd540) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:27:50.178690 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Volume volume-6110d45f-af58-408f-88e7-bd46665d58c3 (6110d45f-af58-408f-88e7-bd46665d58c3): created successfully Oct 01 21:27:50.182121 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (412c5e55-8978-4587-9030-7a80595cd540) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:27:50.187430 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Flow 'volume_create_manager' (5bcce63c-1f53-424a-9569-29f8482971ff) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:27:50.194971 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1e854a98-13d2-415d-9028-78da389b3118 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Created volume successfully. Oct 01 21:27:51.237610 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=62,cinder:INSERT=8,cinder:UPDATE=28,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:27:51.252249 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ea9c75b6-7fee-45a4-bb61-b20881a0a430 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:52.355756 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ea9c75b6-7fee-45a4-bb61-b20881a0a430 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Create snapshot completed successfully Oct 01 21:27:53.408293 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a2351779-fa9c-428f-86e3-8bbca1d11470 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Acquiring lock "cinder-96e357da-fa18-449c-aff8-e6a31e8d4a75-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:27:53.416906 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a2351779-fa9c-428f-86e3-8bbca1d11470 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Lock "cinder-96e357da-fa18-449c-aff8-e6a31e8d4a75-delete_snapshot" acquired by "delete_snapshot" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:27:53.419060 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-a2351779-fa9c-428f-86e3-8bbca1d11470 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:54.422710 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-a2351779-fa9c-428f-86e3-8bbca1d11470 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Created reservations ['012a0b50-7a5e-4d80-be0b-01749a97b1f1', '7dcdccaa-693b-46f8-a2d5-6c947926f8c6', 'b572114a-15a5-46a2-8a7d-f0ab830b26bd', '5311d854-1dcd-4a9e-8641-f8c392d5a0f9'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:27:54.463561 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-a2351779-fa9c-428f-86e3-8bbca1d11470 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Delete snapshot completed successfully. Oct 01 21:27:54.474567 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-a2351779-fa9c-428f-86e3-8bbca1d11470 tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Lock "cinder-96e357da-fa18-449c-aff8-e6a31e8d4a75-delete_snapshot" released by "delete_snapshot" :: held 1.058s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:27:54.523479 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Acquiring lock "cinder-6110d45f-af58-408f-88e7-bd46665d58c3-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:27:54.534376 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Lock "cinder-6110d45f-af58-408f-88e7-bd46665d58c3-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:27:54.575481 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:54.615434 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:27:54.615846 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Volume volume-6110d45f-af58-408f-88e7-bd46665d58c3 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:27:54.622055 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] deleting rbd volume volume-6110d45f-af58-408f-88e7-bd46665d58c3 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:27:54.780807 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Created reservations ['07487ef7-c1e8-4309-be1d-c95430576b76', '57021527-d3b9-4ba8-bd84-c7c857747d80', '779d1509-0cb4-4830-b4f5-8b45aad8367f', '6f12b762-3317-45de-8658-f4086625d0ee'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:27:54.823401 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:54.849706 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:27:54.873267 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:27:54.878637 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Deleted volume successfully. Oct 01 21:27:54.890668 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c33f956d-9501-4033-92a0-6e1804d327de tempest-VolumesSnapshotNegativeTestJSON-1032087441 None] Lock "cinder-6110d45f-af58-408f-88e7-bd46665d58c3-delete_volume" released by "delete_volume" :: held 0.356s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:28:05.277962 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=52,cinder:UPDATE=31,cinder:INSERT=6,cinder:DELETE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:14.038821 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-1878f4aa-98d0-41f4-aa20-37ee1e10682b tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['2feda164-e640-41fe-82b8-4dd6ce6218bc', '0f8e6bd0-6aac-4967-a6e9-a087e6232732', '4015b8ca-f277-4a78-bf24-e25da74c5973', '1b58c628-403a-4f64-a701-443d41596c23'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:28:14.096460 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-1878f4aa-98d0-41f4-aa20-37ee1e10682b tempest-VolumesSnapshotTestJSON-1103874744 None] Delete snapshot completed successfully. Oct 01 21:28:14.104105 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-1878f4aa-98d0-41f4-aa20-37ee1e10682b tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-52f8aea8-115f-4d52-8466-cead4319e408-delete_snapshot" released by "delete_snapshot" :: held 30.208s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:28:15.911968 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-b6c8b6a9-a677-4211-9047-960cdcf1843a req-24553b14-9711-46e5-af5e-d932cfccb809 tempest-VolumesSnapshotTestJSON-1103874744 None] Terminate volume connection completed successfully. Oct 01 21:28:15.912584 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-b6c8b6a9-a677-4211-9047-960cdcf1843a req-24553b14-9711-46e5-af5e-d932cfccb809 tempest-VolumesSnapshotTestJSON-1103874744 None] Deleting attachment c4595c42-725d-488a-8404-28d1e5cf986c. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:28:20.667426 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:28:20.687681 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (360c5628-d7d3-4780-808e-fd1766878de0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:28:20.694239 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (496204fe-83f2-4d1d-a2a0-23954b68ca7a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:20.712444 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (496204fe-83f2-4d1d-a2a0-23954b68ca7a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:28:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-2009988139',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='dd6e51791d934c73bf7d92e7ed1a8909',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:28:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:28:21Z,use_quota=True,user_id='98361240b014405cb20cf9c145061be6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:20.717687 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0cf6241e-4be2-4b9f-8736-39ad159668b6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:20.722537 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0cf6241e-4be2-4b9f-8736-39ad159668b6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:20.727431 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7651e507-5d0f-4212-bb07-64db5ec6bc1b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:20.731353 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7651e507-5d0f-4212-bb07-64db5ec6bc1b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35', 'volume_name': 'volume-bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:20.736265 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8994949b-2124-41e7-a2bd-744db3fa6dc6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:20.740051 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8994949b-2124-41e7-a2bd-744db3fa6dc6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:20.744570 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2aac93fb-dba3-4057-892a-7006f4d186b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:20.745923 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Volume bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35', 'volume_size': 1} Oct 01 21:28:20.746057 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] creating volume 'volume-bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:28:20.747056 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:20.801906 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:28:20.802281 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:28:20.805850 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2aac93fb-dba3-4057-892a-7006f4d186b9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:20.810385 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8ee86f67-7b79-47ee-8aa4-2bf89cd35080) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:20.820749 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Volume volume-bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35 (bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35): created successfully Oct 01 21:28:20.824113 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8ee86f67-7b79-47ee-8aa4-2bf89cd35080) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:20.829891 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (360c5628-d7d3-4780-808e-fd1766878de0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:28:20.838527 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-536e13c7-f03a-4921-a38d-d40dde9d4fde tempest-VolumesSnapshotTestJSON-1103874744 None] Created volume successfully. Oct 01 21:28:21.876262 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=48,cinder:INSERT=8,cinder:UPDATE=22,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:21.902992 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-30f4eddc-b6f4-446e-93eb-d277633a4e9b tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:22.657730 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-30f4eddc-b6f4-446e-93eb-d277633a4e9b tempest-VolumesSnapshotTestJSON-1103874744 None] Create snapshot completed successfully Oct 01 21:28:23.272370 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:28:23.298989 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (19571884-6b11-4002-8335-0d5f43d8c49d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:28:23.305253 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (23bffa4d-fab5-493b-8b46-068ae195a2ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:23.322250 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (23bffa4d-fab5-493b-8b46-068ae195a2ff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:28:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-412811098',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4d16bd29-81bf-41e3-bc86-343bb1565ead,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='dd6e51791d934c73bf7d92e7ed1a8909',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:28:23Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=62586ae0-8829-49cc-8674-69a1d9943951,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:28:23Z,use_quota=True,user_id='98361240b014405cb20cf9c145061be6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:23.328139 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1c0036ae-9a04-4dd1-8e6a-6eee1786ed1a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:23.332169 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1c0036ae-9a04-4dd1-8e6a-6eee1786ed1a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:23.337083 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1db57a07-534a-4d02-b33d-210067cb0f45) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:23.341068 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1db57a07-534a-4d02-b33d-210067cb0f45) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '4d16bd29-81bf-41e3-bc86-343bb1565ead', 'volume_name': 'volume-4d16bd29-81bf-41e3-bc86-343bb1565ead', 'volume_size': 2, 'snapshot_id': '62586ae0-8829-49cc-8674-69a1d9943951'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:23.345773 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (87214cee-97c9-46b8-ad59-85ee77fbb315) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:23.349572 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (87214cee-97c9-46b8-ad59-85ee77fbb315) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:23.354114 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (db06a578-5cb6-4f31-96ff-033672ece188) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:23.355189 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Volume 4d16bd29-81bf-41e3-bc86-343bb1565ead: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-4d16bd29-81bf-41e3-bc86-343bb1565ead', 'volume_size': 2, 'snapshot_id': '62586ae0-8829-49cc-8674-69a1d9943951'} Oct 01 21:28:23.374924 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] cloning volumes/volume-bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35@snapshot-62586ae0-8829-49cc-8674-69a1d9943951 to volume-4d16bd29-81bf-41e3-bc86-343bb1565ead {{(pid=133391) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Oct 01 21:28:23.375567 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:23.410030 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:23.493368 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:28:23.493744 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:28:23.498175 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:23.587139 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (db06a578-5cb6-4f31-96ff-033672ece188) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4d16bd29-81bf-41e3-bc86-343bb1565ead', 'volume_size': 2, 'snapshot_id': '62586ae0-8829-49cc-8674-69a1d9943951'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:23.591544 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0222f874-c608-4174-a678-0c63b1c7cb17) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:23.599950 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Volume volume-4d16bd29-81bf-41e3-bc86-343bb1565ead (4d16bd29-81bf-41e3-bc86-343bb1565ead): created successfully Oct 01 21:28:23.603686 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0222f874-c608-4174-a678-0c63b1c7cb17) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:23.609788 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (19571884-6b11-4002-8335-0d5f43d8c49d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:28:23.622201 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-777c8d23-49bb-44bf-b737-af22586bc326 tempest-VolumesSnapshotTestJSON-1103874744 None] Created volume successfully. Oct 01 21:28:23.836097 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:28:23.836853 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:23.861674 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:23.886753 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:28:24.383628 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9af1a521-dffb-403f-a7e7-8783613fed51 tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-4d16bd29-81bf-41e3-bc86-343bb1565ead-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:28:24.392970 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9af1a521-dffb-403f-a7e7-8783613fed51 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-4d16bd29-81bf-41e3-bc86-343bb1565ead-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:28:24.426019 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9af1a521-dffb-403f-a7e7-8783613fed51 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:24.473275 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9af1a521-dffb-403f-a7e7-8783613fed51 tempest-VolumesSnapshotTestJSON-1103874744 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:28:24.478174 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9af1a521-dffb-403f-a7e7-8783613fed51 tempest-VolumesSnapshotTestJSON-1103874744 None] deleting rbd volume volume-4d16bd29-81bf-41e3-bc86-343bb1565ead {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:28:24.641219 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-9af1a521-dffb-403f-a7e7-8783613fed51 tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['4e4ae1fe-31e1-4e5b-918c-a911e70e8b93', 'ae3b2803-4157-4374-bf2b-be985f9c9036', 'e440b66f-af3d-4e33-86ae-0c7c3910339f', '2c15a5b1-38af-4fb3-b1f8-1fe9ddeffd86'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:28:24.683880 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9af1a521-dffb-403f-a7e7-8783613fed51 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:24.709413 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-9af1a521-dffb-403f-a7e7-8783613fed51 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:24.733797 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-9af1a521-dffb-403f-a7e7-8783613fed51 tempest-VolumesSnapshotTestJSON-1103874744 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:28:24.737716 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-9af1a521-dffb-403f-a7e7-8783613fed51 tempest-VolumesSnapshotTestJSON-1103874744 None] Deleted volume successfully. Oct 01 21:28:24.749624 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-9af1a521-dffb-403f-a7e7-8783613fed51 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-4d16bd29-81bf-41e3-bc86-343bb1565ead-delete_volume" released by "delete_volume" :: held 0.356s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:28:25.783334 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:28:25.802239 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (4d102186-0e7d-48ff-b0c4-8fea8554e83d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:28:25.808447 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5be1b5f9-1844-415d-a04e-1be4e27b6ba7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:25.824234 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5be1b5f9-1844-415d-a04e-1be4e27b6ba7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:28:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-731538317',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=18bdf3d8-c53a-4fb8-8379-542af8fab24d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='dd6e51791d934c73bf7d92e7ed1a8909',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:28:26Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:28:26Z,use_quota=True,user_id='98361240b014405cb20cf9c145061be6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:25.829392 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8248b970-e022-4bf7-8db0-638f18e0c246) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:25.833400 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8248b970-e022-4bf7-8db0-638f18e0c246) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:25.838279 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8619ae4b-0587-4354-a655-48ce473fc3df) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:25.842211 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8619ae4b-0587-4354-a655-48ce473fc3df) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '18bdf3d8-c53a-4fb8-8379-542af8fab24d', 'volume_name': 'volume-18bdf3d8-c53a-4fb8-8379-542af8fab24d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:25.846994 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (56d97fe3-9762-4e23-a76e-92ef387697cf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:25.850757 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (56d97fe3-9762-4e23-a76e-92ef387697cf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:25.855300 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c4b0cae1-422b-4562-b01c-5750bb29db65) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:25.856334 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Volume 18bdf3d8-c53a-4fb8-8379-542af8fab24d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-18bdf3d8-c53a-4fb8-8379-542af8fab24d', 'volume_size': 1} Oct 01 21:28:25.856724 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] creating volume 'volume-18bdf3d8-c53a-4fb8-8379-542af8fab24d' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:28:25.857496 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:25.912945 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:28:25.913319 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:28:25.916934 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c4b0cae1-422b-4562-b01c-5750bb29db65) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-18bdf3d8-c53a-4fb8-8379-542af8fab24d', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:25.922130 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b229521d-ad50-4ff5-a8b1-088acc2cbd0e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:25.931692 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Volume volume-18bdf3d8-c53a-4fb8-8379-542af8fab24d (18bdf3d8-c53a-4fb8-8379-542af8fab24d): created successfully Oct 01 21:28:25.935275 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b229521d-ad50-4ff5-a8b1-088acc2cbd0e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:25.941114 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (4d102186-0e7d-48ff-b0c4-8fea8554e83d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:28:25.949162 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d93466e3-3f6e-4e09-a84e-4996b3ac059e tempest-VolumesSnapshotTestJSON-1103874744 None] Created volume successfully. Oct 01 21:28:26.966300 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-37e5c89f-b3e1-4e54-9756-322487ff7e6c tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:27.747956 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-37e5c89f-b3e1-4e54-9756-322487ff7e6c tempest-VolumesSnapshotTestJSON-1103874744 None] Create snapshot completed successfully Oct 01 21:28:28.342417 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:28:28.369606 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (32c04542-7058-4929-abbc-80f2f002add7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:28:28.376671 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c5aba513-e7cf-4d17-925d-4e7465571ae2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:28.393297 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c5aba513-e7cf-4d17-925d-4e7465571ae2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:28:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1502403663',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f315f605-8d45-4c70-9f17-b3c0f04d5611,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='dd6e51791d934c73bf7d92e7ed1a8909',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:28:28Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=03a182c3-4bf7-49c9-935f-e1aa1f2879c1,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:28:28Z,use_quota=True,user_id='98361240b014405cb20cf9c145061be6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:28.398361 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a75cb056-bbc5-4c24-a34f-e1c7ee7886f5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:28.402321 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a75cb056-bbc5-4c24-a34f-e1c7ee7886f5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:28.407228 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0893801a-14ab-4f8c-be64-f45b2da83821) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:28.411078 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0893801a-14ab-4f8c-be64-f45b2da83821) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'f315f605-8d45-4c70-9f17-b3c0f04d5611', 'volume_name': 'volume-f315f605-8d45-4c70-9f17-b3c0f04d5611', 'volume_size': 1, 'snapshot_id': '03a182c3-4bf7-49c9-935f-e1aa1f2879c1'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:28.415728 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b0a8c28b-1dd3-4896-9583-84d8f9f951c2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:28.419741 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b0a8c28b-1dd3-4896-9583-84d8f9f951c2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:28.424223 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (40405ca2-2880-4184-b810-8030a897cf91) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:28.425250 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Volume f315f605-8d45-4c70-9f17-b3c0f04d5611: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-f315f605-8d45-4c70-9f17-b3c0f04d5611', 'volume_size': 1, 'snapshot_id': '03a182c3-4bf7-49c9-935f-e1aa1f2879c1'} Oct 01 21:28:28.444813 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] cloning volumes/volume-18bdf3d8-c53a-4fb8-8379-542af8fab24d@snapshot-03a182c3-4bf7-49c9-935f-e1aa1f2879c1 to volume-f315f605-8d45-4c70-9f17-b3c0f04d5611 {{(pid=133391) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Oct 01 21:28:28.445437 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:28.489313 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:28.578755 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:28:28.579079 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:28:28.598010 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (40405ca2-2880-4184-b810-8030a897cf91) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f315f605-8d45-4c70-9f17-b3c0f04d5611', 'volume_size': 1, 'snapshot_id': '03a182c3-4bf7-49c9-935f-e1aa1f2879c1'}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:28.602482 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cd50368f-b6cf-48ab-884e-b7e100b7dee1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:28:28.611274 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Volume volume-f315f605-8d45-4c70-9f17-b3c0f04d5611 (f315f605-8d45-4c70-9f17-b3c0f04d5611): created successfully Oct 01 21:28:28.614626 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cd50368f-b6cf-48ab-884e-b7e100b7dee1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:28:28.619825 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Flow 'volume_create_manager' (32c04542-7058-4929-abbc-80f2f002add7) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:28:28.632525 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-dcd41e49-d76b-46e7-b50d-ac5a2465f854 tempest-VolumesSnapshotTestJSON-1103874744 None] Created volume successfully. Oct 01 21:28:29.456906 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6cca2762-ae44-41fe-af43-cc08cfe56170 tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-f315f605-8d45-4c70-9f17-b3c0f04d5611-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:28:29.467904 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6cca2762-ae44-41fe-af43-cc08cfe56170 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-f315f605-8d45-4c70-9f17-b3c0f04d5611-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:28:29.506871 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6cca2762-ae44-41fe-af43-cc08cfe56170 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:29.553723 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6cca2762-ae44-41fe-af43-cc08cfe56170 tempest-VolumesSnapshotTestJSON-1103874744 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:28:29.558545 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6cca2762-ae44-41fe-af43-cc08cfe56170 tempest-VolumesSnapshotTestJSON-1103874744 None] deleting rbd volume volume-f315f605-8d45-4c70-9f17-b3c0f04d5611 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:28:29.719775 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-6cca2762-ae44-41fe-af43-cc08cfe56170 tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['bb31e3f3-4293-4679-843e-df9e213ea868', '2bb2fc09-511b-477c-b51f-6c9ceddbb157', 'babd873f-4055-4c07-9dcc-3c71dd26833e', '1269c0e1-7e44-4a33-a749-757a2dbbf699'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:28:29.744351 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=68,cinder:UPDATE=25,cinder:DELETE=4,cinder:INSERT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:29.764469 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6cca2762-ae44-41fe-af43-cc08cfe56170 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:29.788487 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-6cca2762-ae44-41fe-af43-cc08cfe56170 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:29.812728 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-6cca2762-ae44-41fe-af43-cc08cfe56170 tempest-VolumesSnapshotTestJSON-1103874744 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:28:29.816720 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-6cca2762-ae44-41fe-af43-cc08cfe56170 tempest-VolumesSnapshotTestJSON-1103874744 None] Deleted volume successfully. Oct 01 21:28:29.829009 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-6cca2762-ae44-41fe-af43-cc08cfe56170 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-f315f605-8d45-4c70-9f17-b3c0f04d5611-delete_volume" released by "delete_volume" :: held 0.361s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:28:30.612769 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3d72a834-abe0-4231-9975-4407eb5c1ad8 tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-03a182c3-4bf7-49c9-935f-e1aa1f2879c1-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:28:30.625283 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3d72a834-abe0-4231-9975-4407eb5c1ad8 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-03a182c3-4bf7-49c9-935f-e1aa1f2879c1-delete_snapshot" acquired by "delete_snapshot" :: waited 0.012s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:28:30.627031 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3d72a834-abe0-4231-9975-4407eb5c1ad8 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:31.432009 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3d72a834-abe0-4231-9975-4407eb5c1ad8 tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['f13cd79d-4472-42b2-87b0-b4495462d6e3', '40a90146-7f53-4a8b-bd02-a338cd33d144', '53fbb1d8-8193-4c4b-b02c-4e69391a6e70', 'ba21062b-9040-4764-ac7d-2e02f1705dab'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:28:31.435170 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=60,cinder:DELETE=3,cinder:UPDATE=30,cinder:INSERT=7 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:31.470482 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3d72a834-abe0-4231-9975-4407eb5c1ad8 tempest-VolumesSnapshotTestJSON-1103874744 None] Delete snapshot completed successfully. Oct 01 21:28:31.474584 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3d72a834-abe0-4231-9975-4407eb5c1ad8 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-03a182c3-4bf7-49c9-935f-e1aa1f2879c1-delete_snapshot" released by "delete_snapshot" :: held 0.849s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:28:31.710277 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-18bdf3d8-c53a-4fb8-8379-542af8fab24d-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:28:31.719364 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-18bdf3d8-c53a-4fb8-8379-542af8fab24d-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:28:31.752368 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:31.787147 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:28:31.787567 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] Volume volume-18bdf3d8-c53a-4fb8-8379-542af8fab24d is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:28:31.791226 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] deleting rbd volume volume-18bdf3d8-c53a-4fb8-8379-542af8fab24d {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:28:31.951233 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['2611616a-a8d1-433a-976a-88358edea9a5', 'ded64694-3887-4b62-bb4f-f671c7e8ea63', '476d30d5-9e65-4266-a623-a0e545174cdf', 'c4495ba3-577a-4bec-aa8a-58f7bfb37ad6'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:28:31.994927 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:32.018297 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:32.048384 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:28:32.052519 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] Deleted volume successfully. Oct 01 21:28:32.064217 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-f9712af9-3e99-4845-8091-4994ebefe715 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-18bdf3d8-c53a-4fb8-8379-542af8fab24d-delete_volume" released by "delete_volume" :: held 0.345s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:28:32.870837 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-387015c9-5b4c-4252-9253-cce74c5105f3 tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-62586ae0-8829-49cc-8674-69a1d9943951-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:28:32.881030 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-387015c9-5b4c-4252-9253-cce74c5105f3 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-62586ae0-8829-49cc-8674-69a1d9943951-delete_snapshot" acquired by "delete_snapshot" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:28:32.883777 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-387015c9-5b4c-4252-9253-cce74c5105f3 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:33.433834 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-387015c9-5b4c-4252-9253-cce74c5105f3 tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['ed293040-8526-45c2-ac6a-88dfae4273a1', '501802f9-2cca-4b71-940a-be0ec9d079ca', 'c50a55ac-eb8b-4496-b816-8b191e40e273', 'cf0db998-5433-473b-b03f-ba2f2658ed58'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:28:33.467827 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-387015c9-5b4c-4252-9253-cce74c5105f3 tempest-VolumesSnapshotTestJSON-1103874744 None] Delete snapshot completed successfully. Oct 01 21:28:33.472839 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-387015c9-5b4c-4252-9253-cce74c5105f3 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-62586ae0-8829-49cc-8674-69a1d9943951-delete_snapshot" released by "delete_snapshot" :: held 0.592s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:28:33.974615 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:28:33.983478 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:28:34.014839 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:34.053710 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:28:34.054140 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] Volume volume-bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:28:34.058369 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] deleting rbd volume volume-bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:28:34.212523 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['642eb29a-089d-43d1-8835-aa3398a827bd', '77001040-f6df-4fe9-8a7a-e06d6a2847b5', 'ec8b0d62-24a8-474b-8900-5ca86202a234', 'ffda1de2-d0bb-4ee4-aa29-aea422ddad1b'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:28:34.257819 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:34.283833 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:34.308581 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:28:34.312565 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] Deleted volume successfully. Oct 01 21:28:34.325175 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-435c8883-dbf3-4deb-b840-4ffcfb095bae tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-bf8a63e9-01fd-4c6d-8c61-43c5d7f57f35-delete_volume" released by "delete_volume" :: held 0.342s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:28:35.255678 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-7dea6e35-1c10-40b3-8361-2f40a38aaad1-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:28:35.264313 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-7dea6e35-1c10-40b3-8361-2f40a38aaad1-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:28:35.313821 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:35.383224 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:28:35.390872 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] deleting rbd volume volume-7dea6e35-1c10-40b3-8361-2f40a38aaad1 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:28:35.580894 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:28:35.612102 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] deleting parent snapshot volume-7dea6e35-1c10-40b3-8361-2f40a38aaad1.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:28:36.452256 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['1d253c36-7723-494d-b2c8-9b0607813e86', '4bf691d1-2878-406f-a221-a51fec1503bf', '77827dc3-cceb-4161-84ff-ca72a62c0e72', 'b258bf5a-1660-4043-916b-9d0b1dd34ed1'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:28:36.473980 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=54,cinder:INSERT=11,cinder:UPDATE=34,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:36.505224 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:36.532051 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:36.562765 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:28:36.574524 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] Deleted volume successfully. Oct 01 21:28:36.587289 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-b531f624-d21e-4bef-84d7-b34e02ecc87f tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-7dea6e35-1c10-40b3-8361-2f40a38aaad1-delete_volume" released by "delete_volume" :: held 1.323s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:28:37.465945 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d0fef775-35a3-4599-882f-f7e970d9369f tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-24b3dee7-ae24-428a-8d45-d1e94d233a60-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:28:37.476942 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d0fef775-35a3-4599-882f-f7e970d9369f tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-24b3dee7-ae24-428a-8d45-d1e94d233a60-delete_snapshot" acquired by "delete_snapshot" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:28:37.478718 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d0fef775-35a3-4599-882f-f7e970d9369f tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:38.499287 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-d0fef775-35a3-4599-882f-f7e970d9369f tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['62da4383-be3c-4899-a1bc-f8cf979ccdd3', 'fb18082d-40f4-4409-9e40-3b5de7cc76e6', '67425083-15ad-40af-a8e2-ea69800b9027', '6e162053-37e9-4905-bd65-8573678c6fbc'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:28:38.515141 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=34,cinder:SELECT=57,cinder:INSERT=8,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:28:38.555234 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d0fef775-35a3-4599-882f-f7e970d9369f tempest-VolumesSnapshotTestJSON-1103874744 None] Delete snapshot completed successfully. Oct 01 21:28:38.559988 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d0fef775-35a3-4599-882f-f7e970d9369f tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-24b3dee7-ae24-428a-8d45-d1e94d233a60-delete_snapshot" released by "delete_snapshot" :: held 1.083s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:28:39.612345 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] Acquiring lock "cinder-cda29501-6dd8-4f2b-ba75-8cb61dc80464-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:28:39.623948 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-cda29501-6dd8-4f2b-ba75-8cb61dc80464-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:28:39.657988 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:39.699107 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:28:39.699554 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] Volume volume-cda29501-6dd8-4f2b-ba75-8cb61dc80464 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:28:39.704069 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] deleting rbd volume volume-cda29501-6dd8-4f2b-ba75-8cb61dc80464 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:28:39.907993 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] Created reservations ['e69fd4ed-bb0a-4968-bb47-d801207843dc', '8861cc19-3025-4cf8-9a24-04782e79cad7', '95e9b37b-b6d9-448c-8cc4-08a6fe4d9770', '9f562cdb-876f-4a87-a0b7-0ed45b817650'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:28:39.953198 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:39.978340 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:28:40.004254 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:28:40.009391 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] Deleted volume successfully. Oct 01 21:28:40.022987 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-bfd3c7b4-c3c9-4799-b3ca-23005705f331 tempest-VolumesSnapshotTestJSON-1103874744 None] Lock "cinder-cda29501-6dd8-4f2b-ba75-8cb61dc80464-delete_volume" released by "delete_volume" :: held 0.399s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:28:55.282366 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=10,cinder:SELECT=13,cinder:INSERT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:29:15.287276 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=22,cinder:DELETE=2,cinder:SELECT=31,cinder:INSERT=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:29:23.841838 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:29:23.842532 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:29:23.866459 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:29:23.891032 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:29:25.299395 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:29:30.841592 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-d408449e-eaeb-4383-ba1b-d08421f3395e req-77eb3559-49c5-4f9c-9cfe-eefd39f9e2f5 tempest-TestVolumeBootPattern-1545759599 None] Terminate volume connection completed successfully. Oct 01 21:29:30.842198 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-d408449e-eaeb-4383-ba1b-d08421f3395e req-77eb3559-49c5-4f9c-9cfe-eefd39f9e2f5 tempest-TestVolumeBootPattern-1545759599 None] Deleting attachment 285c8113-fde3-465b-82bd-c6fe44ac0492. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:29:32.811447 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-089a7feb-3cb1-4813-8e1a-568060f129aa tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-a61c69d6-7d79-413d-87ad-e254e121f623-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:29:32.820681 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-089a7feb-3cb1-4813-8e1a-568060f129aa tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-a61c69d6-7d79-413d-87ad-e254e121f623-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:29:32.851897 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-089a7feb-3cb1-4813-8e1a-568060f129aa tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:29:32.922273 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-089a7feb-3cb1-4813-8e1a-568060f129aa tempest-TestVolumeBootPattern-1545759599 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:29:32.928171 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-089a7feb-3cb1-4813-8e1a-568060f129aa tempest-TestVolumeBootPattern-1545759599 None] deleting rbd volume volume-a61c69d6-7d79-413d-87ad-e254e121f623 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:29:33.115093 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-089a7feb-3cb1-4813-8e1a-568060f129aa tempest-TestVolumeBootPattern-1545759599 None] Created reservations ['1333ab87-e0de-40bf-815d-0f940eeddbea', 'c83e530f-4582-473d-bbb9-f6302d77ad45', 'acb7d422-09ae-4c58-b625-303c30fa6fd8', '8a9049d8-5f09-4a41-aeb8-8af647507afe'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:29:33.161906 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-089a7feb-3cb1-4813-8e1a-568060f129aa tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:29:33.186145 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-089a7feb-3cb1-4813-8e1a-568060f129aa tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:29:33.210888 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-089a7feb-3cb1-4813-8e1a-568060f129aa tempest-TestVolumeBootPattern-1545759599 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:29:33.215463 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-089a7feb-3cb1-4813-8e1a-568060f129aa tempest-TestVolumeBootPattern-1545759599 None] Deleted volume successfully. Oct 01 21:29:33.226621 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-089a7feb-3cb1-4813-8e1a-568060f129aa tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-a61c69d6-7d79-413d-87ad-e254e121f623-delete_volume" released by "delete_volume" :: held 0.406s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:29:33.943874 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4e055285-a72a-4d92-9acb-6785798d3b1d tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-56b61d13-1043-4d8c-b993-cc2ef8a3f3ec-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:29:33.952805 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4e055285-a72a-4d92-9acb-6785798d3b1d tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-56b61d13-1043-4d8c-b993-cc2ef8a3f3ec-delete_snapshot" acquired by "delete_snapshot" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:29:33.954376 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-4e055285-a72a-4d92-9acb-6785798d3b1d tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:29:45.305424 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=18,cinder:INSERT=2,cinder:UPDATE=10 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:29:45.309194 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=17,cinder:UPDATE=7,cinder:INSERT=2,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:30:04.103783 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-4e055285-a72a-4d92-9acb-6785798d3b1d tempest-TestVolumeBootPattern-1545759599 None] Created reservations ['4ad21cdc-2bf0-4ad1-b2d1-c33ae7deb543', '927a08fe-9f49-4a53-b394-6033f7c1a392', 'd242ae7e-1c1e-4ec4-98c6-1f8cba00e84c', 'f25dad20-4687-435d-95c3-7e23edd0aa3a'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:30:04.142964 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-4e055285-a72a-4d92-9acb-6785798d3b1d tempest-TestVolumeBootPattern-1545759599 None] Delete snapshot completed successfully. Oct 01 21:30:04.147339 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-4e055285-a72a-4d92-9acb-6785798d3b1d tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-56b61d13-1043-4d8c-b993-cc2ef8a3f3ec-delete_snapshot" released by "delete_snapshot" :: held 30.195s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:30:08.124289 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-b2857e25-cfdf-465c-b7df-b332c2b8a31a req-38939229-0d06-4e64-b12c-5fc678ccca06 tempest-TestVolumeBootPattern-1545759599 None] Terminate volume connection completed successfully. Oct 01 21:30:08.124904 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-b2857e25-cfdf-465c-b7df-b332c2b8a31a req-38939229-0d06-4e64-b12c-5fc678ccca06 tempest-TestVolumeBootPattern-1545759599 None] Deleting attachment c8780623-6e28-49b2-811b-93c327b0aae5. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:30:10.293951 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] Acquiring lock "cinder-91a33031-458c-4c6f-8fe0-fccc855ffd7f-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:30:10.303399 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-91a33031-458c-4c6f-8fe0-fccc855ffd7f-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:30:10.337956 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:30:10.397272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:30:10.404290 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] deleting rbd volume volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:30:10.588095 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 21:30:10.615004 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] deleting parent snapshot volume-91a33031-458c-4c6f-8fe0-fccc855ffd7f.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 21:30:11.521771 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] Created reservations ['7589ad82-0ce8-413d-b678-aaa4110f71d1', '013ad110-21ca-46e7-aaa8-0362b6da3760', '10786692-2283-4ffe-89ee-7e297f573e9e', '8808eee6-9692-49b3-a7a6-76e54fcf1c07'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:30:11.566163 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:30:11.590185 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:30:11.613762 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:30:11.618920 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] Deleted volume successfully. Oct 01 21:30:11.630881 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-d317c04d-d588-48a3-bc96-c168ab15307f tempest-TestVolumeBootPattern-1545759599 None] Lock "cinder-91a33031-458c-4c6f-8fe0-fccc855ffd7f-delete_volume" released by "delete_volume" :: held 1.327s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:30:23.846815 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:30:23.847545 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:30:23.871103 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:30:23.894782 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:30:25.310849 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=23,cinder:UPDATE=15,cinder:INSERT=5,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:30:25.312383 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=16,cinder:SELECT=33,cinder:INSERT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:30:45.307142 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:31:05.316338 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:31:15.322224 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:31:23.850202 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:31:23.850909 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:31:23.873711 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:31:23.897578 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:31:35.328629 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:31:35.330171 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:31:45.339331 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:32:05.341599 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:32:05.343155 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:32:23.852937 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:32:23.853651 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:32:23.876942 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:32:23.900825 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:32:25.340235 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:32:35.349069 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:32:55.351022 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:05.355863 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:23.855793 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:33:23.856486 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:33:23.881451 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:33:23.907305 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:33:25.361904 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:35.367189 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:33:55.364497 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=1,cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:34:05.370963 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:34:23.852444 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:34:23.853128 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:34:23.875669 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:34:23.900336 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:34:25.378165 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:34:35.385550 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:34:55.389571 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:35:05.396685 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:35:23.855388 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:35:23.856049 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:35:23.879342 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:35:23.902936 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:35:25.400827 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:35:35.406323 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:35:55.415766 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:36:05.417186 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:36:23.858522 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:36:23.859203 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:36:23.885393 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:36:23.911333 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:36:25.424855 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:36:35.430135 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:36:55.439795 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:37:05.446428 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:37:23.857694 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:37:23.858388 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:37:23.882818 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:37:23.909019 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:37:25.450896 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:37:35.456287 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:37:55.469298 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:38:05.467085 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:38:15.474839 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:38:23.853853 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:38:23.854554 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:38:23.878363 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:38:23.902480 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:38:35.474702 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:38:35.477633 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:39:05.487957 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:39:05.488566 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:39:23.857482 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:39:23.858145 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:39:23.881877 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:39:23.904975 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:39:35.500162 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:39:35.501662 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:39:55.512286 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:40:05.509978 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:40:15.511204 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:40:23.861429 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:40:23.862178 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:40:23.886253 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:40:23.910703 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:40:35.508965 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:40:45.516549 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:41:05.524930 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:41:23.866808 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:41:23.867488 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:41:23.894333 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:41:23.924769 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:41:25.527677 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:41:25.528385 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=10,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:41:35.531652 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:41:55.538828 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:42:05.537206 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:42:15.544759 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:42:23.871316 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:42:23.871989 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:42:23.897005 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:42:23.925707 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:42:25.547325 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:42:35.552998 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:42:55.551030 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:42:55.560693 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:43:15.561608 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:43:23.871807 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:43:23.872475 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:43:23.896150 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:43:23.919158 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:43:25.563018 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=2,cinder:SELECT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:43:45.577254 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:44:15.577761 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=10,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:44:15.580517 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:44:23.874786 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:44:23.875443 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:44:23.898980 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:44:23.921739 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:44:35.587594 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:44:45.591220 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:45:05.598374 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:45:15.604551 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:45:23.877092 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:45:23.877778 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:45:23.907464 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:45:23.931301 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:45:35.611234 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:45:35.612795 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:45:55.619345 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:46:15.628005 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:46:23.876670 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:46:23.877350 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:46:23.900757 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:46:23.926931 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:46:25.635436 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:46:45.639543 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:46:55.647171 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:01.149966 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:47:01.169134 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Flow 'volume_create_manager' (5febde82-9619-4a2a-bb37-cd1e121d19f4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:47:01.176548 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (92cff467-48be-48ba-a175-f5e91425cb0e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:47:01.192717 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (92cff467-48be-48ba-a175-f5e91425cb0e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:47:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerActionsTestJSON-volume-2031477087',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=87451133-1fee-4ceb-b64f-d489f8a25985,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ae55f903ff94436b8d907d865581cffd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:47:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:47:01Z,use_quota=True,user_id='87e5b194ad2745b6af1f6ad76dd94881',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:47:01.197865 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fbc6eb71-5a52-4780-b3f6-24fd29bf0254) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:47:01.201895 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fbc6eb71-5a52-4780-b3f6-24fd29bf0254) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:47:01.206904 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5e86fd54-6992-48fc-8c81-015adf37fcc5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:47:01.211579 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5e86fd54-6992-48fc-8c81-015adf37fcc5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '87451133-1fee-4ceb-b64f-d489f8a25985', 'volume_name': 'volume-87451133-1fee-4ceb-b64f-d489f8a25985', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:47:01.216316 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (75ffe95f-d8cd-4cd9-a048-48e43994f978) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:47:01.220090 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (75ffe95f-d8cd-4cd9-a048-48e43994f978) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:47:01.225143 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d00d92e6-1aa5-4b96-b1a7-3066e222cd58) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:47:01.226188 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Volume 87451133-1fee-4ceb-b64f-d489f8a25985: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-87451133-1fee-4ceb-b64f-d489f8a25985', 'volume_size': 1} Oct 01 21:47:01.226614 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] creating volume 'volume-87451133-1fee-4ceb-b64f-d489f8a25985' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:47:01.227383 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:47:01.276832 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:47:01.277210 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:47:01.280947 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d00d92e6-1aa5-4b96-b1a7-3066e222cd58) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-87451133-1fee-4ceb-b64f-d489f8a25985', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:47:01.285988 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (025a35c6-12dd-46bf-81b4-e1a126243858) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:47:01.295103 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Volume volume-87451133-1fee-4ceb-b64f-d489f8a25985 (87451133-1fee-4ceb-b64f-d489f8a25985): created successfully Oct 01 21:47:01.298445 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (025a35c6-12dd-46bf-81b4-e1a126243858) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:47:01.304507 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Flow 'volume_create_manager' (5febde82-9619-4a2a-bb37-cd1e121d19f4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:47:01.312110 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-656d2192-ea99-4787-99ed-31ea6fa733d8 tempest-ServerActionsTestJSON-356248320 None] Created volume successfully. Oct 01 21:47:03.125923 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-a4fed070-2df0-4258-88f1-16f6c80052d7 req-bd673af0-5b30-4387-87e5-fdd616910545 tempest-ServerActionsTestJSON-356248320 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:47:03.869211 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-a4fed070-2df0-4258-88f1-16f6c80052d7 req-bd673af0-5b30-4387-87e5-fdd616910545 tempest-ServerActionsTestJSON-356248320 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.743s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:47:03.869932 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-a4fed070-2df0-4258-88f1-16f6c80052d7 req-bd673af0-5b30-4387-87e5-fdd616910545 tempest-ServerActionsTestJSON-356248320 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-87451133-1fee-4ceb-b64f-d489f8a25985', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '87451133-1fee-4ceb-b64f-d489f8a25985', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:47:03.903751 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-a4fed070-2df0-4258-88f1-16f6c80052d7 req-bd673af0-5b30-4387-87e5-fdd616910545 tempest-ServerActionsTestJSON-356248320 None] Connection info returned from driver {'name': 'volumes/volume-87451133-1fee-4ceb-b64f-d489f8a25985', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '87451133-1fee-4ceb-b64f-d489f8a25985', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '46fc33f2-c81e-468d-ab2d-a2448de0b20e'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:47:03.958819 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-a4fed070-2df0-4258-88f1-16f6c80052d7 req-bd673af0-5b30-4387-87e5-fdd616910545 tempest-ServerActionsTestJSON-356248320 None] attachment_update completed successfully. Oct 01 21:47:15.655276 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=22,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:23.872782 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:47:23.873804 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:47:23.900385 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:47:23.924043 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:47:25.663826 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=17,cinder:UPDATE=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:26.057832 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-942befe8-1224-4d7c-9014-7bc445ee1c6e req-997e0524-b6f8-4ce9-a426-bc12bd1ebb13 tempest-ServerActionsTestJSON-356248320 None] Terminate volume connection completed successfully. Oct 01 21:47:26.058254 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-942befe8-1224-4d7c-9014-7bc445ee1c6e req-997e0524-b6f8-4ce9-a426-bc12bd1ebb13 tempest-ServerActionsTestJSON-356248320 None] Deleting attachment 46fc33f2-c81e-468d-ab2d-a2448de0b20e. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:47:28.418422 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-942befe8-1224-4d7c-9014-7bc445ee1c6e req-fb277d28-7167-44cc-b022-8874d07965b6 tempest-ServerActionsTestJSON-356248320 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:47:29.159921 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-942befe8-1224-4d7c-9014-7bc445ee1c6e req-fb277d28-7167-44cc-b022-8874d07965b6 tempest-ServerActionsTestJSON-356248320 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.741s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:47:29.160788 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-942befe8-1224-4d7c-9014-7bc445ee1c6e req-fb277d28-7167-44cc-b022-8874d07965b6 tempest-ServerActionsTestJSON-356248320 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-87451133-1fee-4ceb-b64f-d489f8a25985', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '87451133-1fee-4ceb-b64f-d489f8a25985', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:47:29.196173 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-942befe8-1224-4d7c-9014-7bc445ee1c6e req-fb277d28-7167-44cc-b022-8874d07965b6 tempest-ServerActionsTestJSON-356248320 None] Connection info returned from driver {'name': 'volumes/volume-87451133-1fee-4ceb-b64f-d489f8a25985', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '87451133-1fee-4ceb-b64f-d489f8a25985', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'a966d490-b5c2-4bc5-95b2-19def59edd6e'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:47:29.254041 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-942befe8-1224-4d7c-9014-7bc445ee1c6e req-fb277d28-7167-44cc-b022-8874d07965b6 tempest-ServerActionsTestJSON-356248320 None] attachment_update completed successfully. Oct 01 21:47:35.624111 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-187e19a9-a7c3-4bf7-b5cf-23be54706ec7 req-afb16ee3-efa3-4e8e-8c64-075c1062ce39 tempest-ServerActionsTestJSON-356248320 None] Terminate volume connection completed successfully. Oct 01 21:47:35.624817 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-187e19a9-a7c3-4bf7-b5cf-23be54706ec7 req-afb16ee3-efa3-4e8e-8c64-075c1062ce39 tempest-ServerActionsTestJSON-356248320 None] Deleting attachment a966d490-b5c2-4bc5-95b2-19def59edd6e. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:47:37.813374 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-187e19a9-a7c3-4bf7-b5cf-23be54706ec7 req-aae59942-8964-4963-8160-2c89f47fea56 tempest-ServerActionsTestJSON-356248320 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:47:38.551806 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-187e19a9-a7c3-4bf7-b5cf-23be54706ec7 req-aae59942-8964-4963-8160-2c89f47fea56 tempest-ServerActionsTestJSON-356248320 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.738s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:47:38.552640 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-187e19a9-a7c3-4bf7-b5cf-23be54706ec7 req-aae59942-8964-4963-8160-2c89f47fea56 tempest-ServerActionsTestJSON-356248320 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-87451133-1fee-4ceb-b64f-d489f8a25985', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '87451133-1fee-4ceb-b64f-d489f8a25985', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:47:38.577427 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-187e19a9-a7c3-4bf7-b5cf-23be54706ec7 req-aae59942-8964-4963-8160-2c89f47fea56 tempest-ServerActionsTestJSON-356248320 None] Connection info returned from driver {'name': 'volumes/volume-87451133-1fee-4ceb-b64f-d489f8a25985', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '87451133-1fee-4ceb-b64f-d489f8a25985', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '482ae14e-5674-4ec0-8d50-00df7fefe5cb'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:47:38.624604 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-187e19a9-a7c3-4bf7-b5cf-23be54706ec7 req-aae59942-8964-4963-8160-2c89f47fea56 tempest-ServerActionsTestJSON-356248320 None] attachment_update completed successfully. Oct 01 21:47:45.010514 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-4201b969-5057-4828-af63-3ef24e3632d9 req-97306430-8c05-49f0-8eb8-6411976a9d9e tempest-ServerActionsTestJSON-356248320 None] Terminate volume connection completed successfully. Oct 01 21:47:45.011156 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-4201b969-5057-4828-af63-3ef24e3632d9 req-97306430-8c05-49f0-8eb8-6411976a9d9e tempest-ServerActionsTestJSON-356248320 None] Deleting attachment 482ae14e-5674-4ec0-8d50-00df7fefe5cb. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:47:45.941437 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] Acquiring lock "cinder-87451133-1fee-4ceb-b64f-d489f8a25985-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:47:45.949634 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] Lock "cinder-87451133-1fee-4ceb-b64f-d489f8a25985-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:47:45.952468 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=29,cinder:UPDATE=6 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:47:45.986053 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:47:46.025649 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:47:46.026318 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] Volume volume-87451133-1fee-4ceb-b64f-d489f8a25985 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:47:46.030714 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] deleting rbd volume volume-87451133-1fee-4ceb-b64f-d489f8a25985 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:47:46.189156 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] Created reservations ['474e0f40-5cef-42d6-ad3a-683dc29b5bf1', 'e26cb1e7-924d-49b9-a180-3f05f5253e5f', 'fe928545-c91f-46f5-a54d-9ab55a615ab2', '762ff516-cdcf-4714-8666-65043c740d8f'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:47:46.231950 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:47:46.257481 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:47:46.296272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:47:46.301083 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] Deleted volume successfully. Oct 01 21:47:46.313252 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-ab3d83f0-a47f-45d8-a94a-2ef8170042df tempest-ServerActionsTestJSON-356248320 None] Lock "cinder-87451133-1fee-4ceb-b64f-d489f8a25985-delete_volume" released by "delete_volume" :: held 0.364s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:48:05.662505 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=51,cinder:UPDATE=12,cinder:INSERT=2,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:15.664814 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=13,cinder:INSERT=2,cinder:UPDATE=9 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:23.870165 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:48:23.870958 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:48:23.902471 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:48:23.929566 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:48:35.671299 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:48:45.679390 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:49:15.679393 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:49:15.686809 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:49:22.279331 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:49:22.299376 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Flow 'volume_create_manager' (af0cc10b-73df-4ad6-9447-cc2301e87bb8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:49:22.306285 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (970b6b6a-b6bd-4e69-aeaf-11a7acbd1443) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:49:22.322394 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (970b6b6a-b6bd-4e69-aeaf-11a7acbd1443) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:49:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-306748653',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5c5d3990-22f7-4f2d-a4eb-af828a87f071,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f02c2d9ca9fa43bdad1329ff8583b742',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:49:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:49:22Z,use_quota=True,user_id='0d1d8f5e9d074c2a89f5cf7c874392ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:49:22.327534 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9fb5cb79-6cb9-44a1-994a-4469489b87d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:49:22.331523 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9fb5cb79-6cb9-44a1-994a-4469489b87d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:49:22.336383 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6eb524e6-1fdc-495f-bbfd-a7c372390940) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:49:22.340291 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6eb524e6-1fdc-495f-bbfd-a7c372390940) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5c5d3990-22f7-4f2d-a4eb-af828a87f071', 'volume_name': 'volume-5c5d3990-22f7-4f2d-a4eb-af828a87f071', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:49:22.344949 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0200b05b-d3eb-4217-bb35-39f69c0d0bec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:49:22.348735 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0200b05b-d3eb-4217-bb35-39f69c0d0bec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:49:22.353196 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cf5db4fa-4678-4557-afbb-57e0c0bf30f6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:49:22.354245 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Volume 5c5d3990-22f7-4f2d-a4eb-af828a87f071: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5c5d3990-22f7-4f2d-a4eb-af828a87f071', 'volume_size': 1} Oct 01 21:49:22.354659 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] creating volume 'volume-5c5d3990-22f7-4f2d-a4eb-af828a87f071' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:49:22.355415 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:49:22.453317 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:49:22.453693 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:49:22.457556 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cf5db4fa-4678-4557-afbb-57e0c0bf30f6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5c5d3990-22f7-4f2d-a4eb-af828a87f071', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:49:22.461950 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c10879b0-633b-491b-8003-30bfd13058d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:49:22.470812 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Volume volume-5c5d3990-22f7-4f2d-a4eb-af828a87f071 (5c5d3990-22f7-4f2d-a4eb-af828a87f071): created successfully Oct 01 21:49:22.474255 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c10879b0-633b-491b-8003-30bfd13058d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:49:22.479538 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Flow 'volume_create_manager' (af0cc10b-73df-4ad6-9447-cc2301e87bb8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:49:22.487829 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-426baf22-d9e8-4a12-ab77-36bc80d0e472 tempest-TestMinimumBasicScenario-172740098 None] Created volume successfully. Oct 01 21:49:23.874481 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:49:23.875174 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:49:23.903114 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:49:23.927154 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:49:24.023296 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-3b7f1755-376c-4a08-aac6-93ed8d78365f req-e3fa0969-f9c8-42e9-bb71-585cf3635b4e tempest-TestMinimumBasicScenario-172740098 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:49:25.118462 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-3b7f1755-376c-4a08-aac6-93ed8d78365f req-e3fa0969-f9c8-42e9-bb71-585cf3635b4e tempest-TestMinimumBasicScenario-172740098 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.095s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:49:25.119413 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-3b7f1755-376c-4a08-aac6-93ed8d78365f req-e3fa0969-f9c8-42e9-bb71-585cf3635b4e tempest-TestMinimumBasicScenario-172740098 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-5c5d3990-22f7-4f2d-a4eb-af828a87f071', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '5c5d3990-22f7-4f2d-a4eb-af828a87f071', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:49:25.144035 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-3b7f1755-376c-4a08-aac6-93ed8d78365f req-e3fa0969-f9c8-42e9-bb71-585cf3635b4e tempest-TestMinimumBasicScenario-172740098 None] Connection info returned from driver {'name': 'volumes/volume-5c5d3990-22f7-4f2d-a4eb-af828a87f071', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '5c5d3990-22f7-4f2d-a4eb-af828a87f071', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '861ec855-92aa-45df-bf0b-c790c03086c7'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:49:25.191346 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-3b7f1755-376c-4a08-aac6-93ed8d78365f req-e3fa0969-f9c8-42e9-bb71-585cf3635b4e tempest-TestMinimumBasicScenario-172740098 None] attachment_update completed successfully. Oct 01 21:49:27.229085 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-5f8fbe7e-5134-4469-bbb3-a32072ea90d6 tempest-TestMinimumBasicScenario-172740098 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:49:27.636502 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-5f8fbe7e-5134-4469-bbb3-a32072ea90d6 tempest-TestMinimumBasicScenario-172740098 None] Create snapshot completed successfully Oct 01 21:49:45.681091 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=25,cinder:UPDATE=6,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:49:55.686813 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=6,cinder:SELECT=25,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:50:15.694095 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:50:23.872568 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:50:23.873240 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:50:23.897181 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:50:23.921275 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:50:25.702127 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:50:45.708545 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:50:55.714182 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:51:15.721039 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:51:19.482268 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-880d03e9-5d7a-47f4-a8d3-2b5e508eb506 tempest-TestMinimumBasicScenario-172740098 None] Acquiring lock "cinder-2e343643-b8be-49e4-986d-d498f1d7db97-delete_snapshot" by "delete_snapshot" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:51:19.495268 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-880d03e9-5d7a-47f4-a8d3-2b5e508eb506 tempest-TestMinimumBasicScenario-172740098 None] Lock "cinder-2e343643-b8be-49e4-986d-d498f1d7db97-delete_snapshot" acquired by "delete_snapshot" :: waited 0.013s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:51:19.496437 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-880d03e9-5d7a-47f4-a8d3-2b5e508eb506 tempest-TestMinimumBasicScenario-172740098 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:51:23.876592 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:51:23.877270 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:51:23.899693 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:51:23.922114 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:51:35.729580 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:51:45.735613 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:51:49.619689 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-880d03e9-5d7a-47f4-a8d3-2b5e508eb506 tempest-TestMinimumBasicScenario-172740098 None] Created reservations ['7cad18cc-34a2-4f3d-ac35-30fb8f3f1656', '89e65fdf-28a1-4c3b-a91d-43df48b0278b', 'aea9fd07-c79b-4f40-bab6-b3bb3cc7dcf2', 'a2694cac-36e0-4c51-841d-dfa8f3419465'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:51:49.655939 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-880d03e9-5d7a-47f4-a8d3-2b5e508eb506 tempest-TestMinimumBasicScenario-172740098 None] Delete snapshot completed successfully. Oct 01 21:51:49.660567 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-880d03e9-5d7a-47f4-a8d3-2b5e508eb506 tempest-TestMinimumBasicScenario-172740098 None] Lock "cinder-2e343643-b8be-49e4-986d-d498f1d7db97-delete_snapshot" released by "delete_snapshot" :: held 30.166s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:52:05.741731 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=17,cinder:UPDATE=4,cinder:INSERT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:52:15.746194 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=10,cinder:INSERT=3,cinder:UPDATE=10 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:52:23.877465 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:52:23.878543 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:52:23.902350 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:52:23.926671 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:52:25.753889 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:52:45.759814 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:52:55.764276 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:53:07.895196 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-85c3576b-153c-49aa-a9cf-ef24e6cf21d8 req-d94cad5d-0a52-4eff-b42a-632f75f31f7a tempest-TestMinimumBasicScenario-172740098 None] Terminate volume connection completed successfully. Oct 01 21:53:07.895790 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-85c3576b-153c-49aa-a9cf-ef24e6cf21d8 req-d94cad5d-0a52-4eff-b42a-632f75f31f7a tempest-TestMinimumBasicScenario-172740098 None] Deleting attachment 861ec855-92aa-45df-bf0b-c790c03086c7. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:53:08.562473 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] Acquiring lock "cinder-5c5d3990-22f7-4f2d-a4eb-af828a87f071-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:53:08.573346 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] Lock "cinder-5c5d3990-22f7-4f2d-a4eb-af828a87f071-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:53:08.610776 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:53:08.646130 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:53:08.646619 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] Volume volume-5c5d3990-22f7-4f2d-a4eb-af828a87f071 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:53:08.651085 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] deleting rbd volume volume-5c5d3990-22f7-4f2d-a4eb-af828a87f071 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:53:08.808321 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] Created reservations ['952c630c-801a-4dc6-9372-837636cd59d5', '4dc7d38e-0047-44b6-8b7c-bd16b060633e', 'fbdab810-dead-44e6-87ab-e00fb5ae9330', '32136386-7e48-4384-a318-b953829d1377'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:53:08.851318 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:53:08.877010 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:53:08.901082 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:53:08.905437 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] Deleted volume successfully. Oct 01 21:53:08.916943 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-c57d9fd4-c8a2-4e78-8673-d422daf2813b tempest-TestMinimumBasicScenario-172740098 None] Lock "cinder-5c5d3990-22f7-4f2d-a4eb-af828a87f071-delete_volume" released by "delete_volume" :: held 0.344s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:53:15.776546 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=20,cinder:UPDATE=6,cinder:INSERT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:53:23.882503 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:53:23.883253 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:53:23.913422 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:53:23.945497 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:53:35.767099 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=17,cinder:UPDATE=12,cinder:INSERT=2,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:53:47.593710 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:53:47.609029 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 21:53:47.628545 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Flow 'volume_create_manager' (0cf598ba-8461-482e-8844-f646cef46736) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:53:47.634779 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (584da8c0-ccdc-4dc4-9ae1-15b312b57e10) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:53:47.651300 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (584da8c0-ccdc-4dc4-9ae1-15b312b57e10) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T21:53:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1128676981',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=9a9dd363-c0ba-48a2-96cc-d36b034cc7b9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f02c2d9ca9fa43bdad1329ff8583b742',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T21:53:48Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T21:53:48Z,use_quota=True,user_id='0d1d8f5e9d074c2a89f5cf7c874392ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:53:47.656499 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c8d5ca1f-a9c6-449f-87d7-4886c31de70f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:53:47.660459 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c8d5ca1f-a9c6-449f-87d7-4886c31de70f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:53:47.665322 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (febf6c97-0140-4339-8df3-d186993680a4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:53:47.669260 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (febf6c97-0140-4339-8df3-d186993680a4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '9a9dd363-c0ba-48a2-96cc-d36b034cc7b9', 'volume_name': 'volume-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:53:47.673960 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5f3dc1dc-4f10-4c79-8796-70108537d8ce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:53:47.677716 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5f3dc1dc-4f10-4c79-8796-70108537d8ce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:53:47.683013 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cffc923a-3e97-46c3-86c7-9a8dacbce554) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:53:47.684049 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Volume 9a9dd363-c0ba-48a2-96cc-d36b034cc7b9: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9', 'volume_size': 1} Oct 01 21:53:47.684450 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] creating volume 'volume-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9' {{(pid=133391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Oct 01 21:53:47.685244 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:53:47.737977 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 21:53:47.738277 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 21:53:47.741831 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cffc923a-3e97-46c3-86c7-9a8dacbce554) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9', 'volume_size': 1}' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:53:47.746787 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e53ff205-5393-4cbc-b80a-20d88475bfa3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 21:53:47.756057 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Volume volume-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9 (9a9dd363-c0ba-48a2-96cc-d36b034cc7b9): created successfully Oct 01 21:53:47.759703 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e53ff205-5393-4cbc-b80a-20d88475bfa3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 21:53:47.765658 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Flow 'volume_create_manager' (0cf598ba-8461-482e-8844-f646cef46736) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 21:53:47.773228 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-ccb0a8f1-e350-4531-a85d-fb415d1eeb14 tempest-TestMinimumBasicScenario-172740098 None] Created volume successfully. Oct 01 21:53:49.227311 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-8d6b9f35-d7b3-4728-9f09-9a7997316ac4 req-fa7da6c2-1ee7-41ea-80f0-6eaeaee46eae tempest-TestMinimumBasicScenario-172740098 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 21:53:50.118385 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-8d6b9f35-d7b3-4728-9f09-9a7997316ac4 req-fa7da6c2-1ee7-41ea-80f0-6eaeaee46eae tempest-TestMinimumBasicScenario-172740098 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.891s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 21:53:50.119258 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-8d6b9f35-d7b3-4728-9f09-9a7997316ac4 req-fa7da6c2-1ee7-41ea-80f0-6eaeaee46eae tempest-TestMinimumBasicScenario-172740098 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '9a9dd363-c0ba-48a2-96cc-d36b034cc7b9', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 21:53:50.145493 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-8d6b9f35-d7b3-4728-9f09-9a7997316ac4 req-fa7da6c2-1ee7-41ea-80f0-6eaeaee46eae tempest-TestMinimumBasicScenario-172740098 None] Connection info returned from driver {'name': 'volumes/volume-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '9a9dd363-c0ba-48a2-96cc-d36b034cc7b9', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'e9069da4-4bf8-4044-946a-23f235b078fe'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 21:53:50.194764 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-8d6b9f35-d7b3-4728-9f09-9a7997316ac4 req-fa7da6c2-1ee7-41ea-80f0-6eaeaee46eae tempest-TestMinimumBasicScenario-172740098 None] attachment_update completed successfully. Oct 01 21:54:15.785389 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=16,cinder:UPDATE=6 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:54:23.885306 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:54:23.886183 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:54:23.912180 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:54:23.935684 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:54:25.791624 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=24,cinder:UPDATE=4,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:54:45.804886 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:54:55.809146 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:05.810012 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:09.158819 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-1574be0f-bbcb-4821-9fe3-16aa75c7bba7 req-6b30a307-d4a9-4a28-ac0b-bdfb3f3c46cf tempest-TestMinimumBasicScenario-172740098 None] Terminate volume connection completed successfully. Oct 01 21:55:09.159504 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-1574be0f-bbcb-4821-9fe3-16aa75c7bba7 req-6b30a307-d4a9-4a28-ac0b-bdfb3f3c46cf tempest-TestMinimumBasicScenario-172740098 None] Deleting attachment e9069da4-4bf8-4044-946a-23f235b078fe. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 21:55:09.793567 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] Acquiring lock "cinder-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 21:55:09.802779 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] Lock "cinder-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 21:55:09.837529 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:55:09.875640 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 21:55:09.876086 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] Volume volume-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9 is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 21:55:09.879988 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] deleting rbd volume volume-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 21:55:10.043720 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] Created reservations ['956e8fce-b9ab-47b3-8e2e-5c7789f1b6cb', 'b69851bb-d126-4add-bc68-c4190d25acc1', '2d3908c8-049a-4c6a-a70a-132a7dcee3ed', 'd96885f4-e393-47b1-b648-63f65d5f91a1'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 21:55:10.087362 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:55:10.110272 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:55:10.134533 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:55:10.139456 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] Deleted volume successfully. Oct 01 21:55:10.152285 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-3542adf3-9231-4f23-8140-5e3bf3b13adc tempest-TestMinimumBasicScenario-172740098 None] Lock "cinder-9a9dd363-c0ba-48a2-96cc-d36b034cc7b9-delete_volume" released by "delete_volume" :: held 0.350s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 21:55:23.890981 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:55:23.891637 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:55:23.916694 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:55:23.947346 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:55:25.812857 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=13,cinder:INSERT=1,cinder:UPDATE=8 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:25.817186 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=22,cinder:UPDATE=10,cinder:INSERT=3,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:55:45.831805 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:56:05.820650 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=9,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:56:23.892213 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:56:23.892919 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:56:23.916252 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:56:23.940714 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:56:25.828933 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:56:35.840906 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:56:55.841968 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:57:05.847043 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:57:23.896890 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:57:23.897657 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:57:23.921078 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:57:23.957507 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:57:25.852719 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:57:35.858069 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:57:55.863333 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:58:05.874102 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:58:23.893389 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:58:23.894063 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:58:23.917615 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:58:23.940737 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:58:35.868533 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:58:35.869812 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:58:55.865963 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:59:05.877376 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:59:15.880689 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:59:23.895451 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 21:59:23.896118 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:59:23.919438 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 21:59:23.943842 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 21:59:35.884713 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 21:59:45.893100 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:00:05.893458 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:00:23.899884 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:00:23.900577 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:00:23.923327 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:00:23.950078 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:00:25.900213 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:00:35.904352 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:00:55.911985 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:01:05.916700 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:01:23.905365 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:01:23.906019 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:01:23.930322 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:01:23.955066 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:01:25.924546 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:01:35.932459 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=2,cinder:SELECT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:01:55.935301 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:02:15.934327 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:02:23.910991 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:02:23.911656 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:02:23.936074 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:02:23.959764 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:02:25.943866 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=2,cinder:SELECT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:02:45.945457 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:03:05.950205 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=3,cinder:SELECT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:03:23.915945 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:03:23.916656 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:03:23.940603 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:03:23.963437 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:03:25.950627 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:03:25.951282 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:03:45.949979 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:03:55.954509 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:04:15.960315 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:04:23.918188 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:04:23.918932 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:04:23.942135 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:04:23.966433 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:04:25.969270 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:04:45.972179 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:04:55.976264 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:05:15.984502 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:05:23.921493 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:05:23.922201 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:05:23.946593 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:05:23.969303 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:05:25.985819 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:05:55.974999 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:06:05.978126 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:06:23.923865 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:06:23.924553 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:06:23.949639 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:06:23.973829 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:06:25.982812 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:06:25.986370 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:06:45.992733 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:06:56.001655 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:07:01.619748 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1368}} Oct 01 22:07:01.639235 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Flow 'volume_create_manager' (61000276-2447-4836-bf51-8e4e6b347361) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 22:07:01.645372 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8111b248-9391-4fe3-8c70-f952fd3f0e08) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 22:07:01.660763 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8111b248-9391-4fe3-8c70-f952fd3f0e08) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-10-01T22:07:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-570161905',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=223aaf60-9290-4c78-830a-bd61587db2a6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b904f70d6c7144db99dd4897d62ec567',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-10-01T22:07:02Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-10-01T22:07:02Z,use_quota=True,user_id='c145fd00464c4dd1bb9e48320d8e916b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(68adf31b-ae02-4f64-af1d-7071badf221b),volume_type_id=68adf31b-ae02-4f64-af1d-7071badf221b)' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 22:07:01.665801 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (91b49ca3-a31e-4a61-92b3-6bd2c3b8a286) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 22:07:01.669679 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (91b49ca3-a31e-4a61-92b3-6bd2c3b8a286) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 22:07:01.674761 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d8b18eff-b070-4697-827c-ba15e9521c9e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 22:07:01.753352 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d8b18eff-b070-4697-827c-ba15e9521c9e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '223aaf60-9290-4c78-830a-bd61587db2a6', 'volume_name': 'volume-223aaf60-9290-4c78-830a-bd61587db2a6', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 22:07:01.758755 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b0d39e10-9e94-478c-880f-408ab5c5ff63) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 22:07:01.762525 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b0d39e10-9e94-478c-880f-408ab5c5ff63) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 22:07:01.767023 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0e3035fb-ec42-4cfb-ada6-82bd77112bad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 22:07:01.768117 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Volume 223aaf60-9290-4c78-830a-bd61587db2a6: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-223aaf60-9290-4c78-830a-bd61587db2a6', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Oct 01 22:07:01.768694 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Cloning 223aaf60-9290-4c78-830a-bd61587db2a6 from image 90f3f35e-76ec-4c31-b581-329b315fe607 at location (None, None). {{(pid=133391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1071}} Oct 01 22:07:01.769439 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Acquiring lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" by "_prepare_image_cache_entry" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 22:07:01.779743 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" acquired by "_prepare_image_cache_entry" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 22:07:01.789182 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 22, 7, 1, 783837)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 22:07:01.789618 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 22:07:01.790009 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Found cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:886}} Oct 01 22:07:01.802441 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Lock "cinder-90f3f35e-76ec-4c31-b581-329b315fe607" released by "_prepare_image_cache_entry" :: held 0.023s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 22:07:01.803376 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Attempting to retrieve cache entry for image = 90f3f35e-76ec-4c31-b581-329b315fe607 on host ncdv-master-1@ceph#ceph. {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:827}} Oct 01 22:07:01.809700 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Found image-volume cache entry: {'id': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'volume_id': '822f6692-9e60-4c53-96e6-c1632fc60252', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16), 'last_used': datetime.datetime(2022, 10, 1, 22, 7, 1, 807528)}. {{(pid=133391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Oct 01 22:07:01.810111 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.image.cache [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Image-volume cache entry image_update_at = 2022-10-01 20:40:16+00:00, requested image updated_at = 2022-10-01 20:40:16+00:00. {{(pid=133391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Oct 01 22:07:01.810516 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Creating from source image-volume 822f6692-9e60-4c53-96e6-c1632fc60252 {{(pid=133391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:841}} Oct 01 22:07:01.822213 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:07:01.893876 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] creating snapshot='volume-223aaf60-9290-4c78-830a-bd61587db2a6.clone_snap' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Oct 01 22:07:02.249883 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] cloning 'volume-822f6692-9e60-4c53-96e6-c1632fc60252@volume-223aaf60-9290-4c78-830a-bd61587db2a6.clone_snap' to 'volume-223aaf60-9290-4c78-830a-bd61587db2a6' {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Oct 01 22:07:02.356043 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Volume volume-ea35f176-da9d-49b3-9ff6-4dce9e05b3c8.deleted is not a clone. {{(pid=133391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Oct 01 22:07:02.357310 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Oct 01 22:07:02.357680 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Oct 01 22:07:02.370909 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] clone created successfully {{(pid=133391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Oct 01 22:07:02.371526 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.volume_utils [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Marking volume 223aaf60-9290-4c78-830a-bd61587db2a6 as bootable. {{(pid=133391) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Oct 01 22:07:02.379406 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Copying metadata from image 90f3f35e-76ec-4c31-b581-329b315fe607 to 223aaf60-9290-4c78-830a-bd61587db2a6. {{(pid=133391) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:452}} Oct 01 22:07:02.379838 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.flows.manager.create_volume [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Creating volume glance metadata for volume 223aaf60-9290-4c78-830a-bd61587db2a6 backed by image 90f3f35e-76ec-4c31-b581-329b315fe607 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133391) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:683}} Oct 01 22:07:02.431154 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0e3035fb-ec42-4cfb-ada6-82bd77112bad) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-223aaf60-9290-4c78-830a-bd61587db2a6', 'volume_size': 1, 'image_id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': '549fa0b49ffd4207961a9dc3298560fa', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '90f3f35e-76ec-4c31-b581-329b315fe607', 'created_at': datetime.datetime(2022, 10, 1, 20, 39, 59, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 10, 1, 20, 40, 16, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/90f3f35e-76ec-4c31-b581-329b315fe607/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 22:07:02.435675 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f3757372-6c91-4e80-bdb0-14960242491c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Oct 01 22:07:02.443544 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.flows.manager.create_volume [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Volume volume-223aaf60-9290-4c78-830a-bd61587db2a6 (223aaf60-9290-4c78-830a-bd61587db2a6): created successfully Oct 01 22:07:02.446918 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f3757372-6c91-4e80-bdb0-14960242491c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Oct 01 22:07:02.452164 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Flow 'volume_create_manager' (61000276-2447-4836-bf51-8e4e6b347361) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Oct 01 22:07:02.459523 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [None req-e7dc7107-96f3-4bae-af4e-60ac59744aff tempest-TestShelveInstance-1011857455 None] Created volume successfully. Oct 01 22:07:04.564020 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-a32b4054-55bb-49e1-a377-f0877e12d4a4 req-325151d8-db7c-4da0-96b0-ccc719b502a5 tempest-TestShelveInstance-1011857455 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 22:07:05.311690 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-a32b4054-55bb-49e1-a377-f0877e12d4a4 req-325151d8-db7c-4da0-96b0-ccc719b502a5 tempest-TestShelveInstance-1011857455 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.747s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 22:07:05.312466 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-a32b4054-55bb-49e1-a377-f0877e12d4a4 req-325151d8-db7c-4da0-96b0-ccc719b502a5 tempest-TestShelveInstance-1011857455 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-223aaf60-9290-4c78-830a-bd61587db2a6', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '223aaf60-9290-4c78-830a-bd61587db2a6', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 22:07:05.338952 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-a32b4054-55bb-49e1-a377-f0877e12d4a4 req-325151d8-db7c-4da0-96b0-ccc719b502a5 tempest-TestShelveInstance-1011857455 None] Connection info returned from driver {'name': 'volumes/volume-223aaf60-9290-4c78-830a-bd61587db2a6', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '223aaf60-9290-4c78-830a-bd61587db2a6', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'a3d6e861-e685-4fc6-afbe-e156b1313e53'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 22:07:05.392321 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-a32b4054-55bb-49e1-a377-f0877e12d4a4 req-325151d8-db7c-4da0-96b0-ccc719b502a5 tempest-TestShelveInstance-1011857455 None] attachment_update completed successfully. Oct 01 22:07:16.004265 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=31,cinder:UPDATE=4,cinder:INSERT=7,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:07:16.005849 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=27,cinder:UPDATE=7,cinder:INSERT=5 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:07:23.920077 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:07:23.921201 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:07:23.947906 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:07:23.973912 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:07:36.005187 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:07:46.013702 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:08:06.010060 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=2,cinder:SELECT=7 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:08:06.012073 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:08:23.919739 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:08:23.920671 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:08:23.943436 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:08:23.967905 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:08:46.011895 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:08:46.012610 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:09:06.009769 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:09:16.019583 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:09:23.922649 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:09:23.923318 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:09:23.945937 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:09:23.969504 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:09:34.116240 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-37f38c8e-fd70-4001-aa6a-98706d72099b req-cdf51806-5343-4fb8-b1aa-14ed96684aab tempest-TestShelveInstance-1011857455 None] Terminate volume connection completed successfully. Oct 01 22:09:34.116893 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-37f38c8e-fd70-4001-aa6a-98706d72099b req-cdf51806-5343-4fb8-b1aa-14ed96684aab tempest-TestShelveInstance-1011857455 None] Deleting attachment a3d6e861-e685-4fc6-afbe-e156b1313e53. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 22:09:36.022249 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=10,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:09:40.572907 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-468ffc46-caf4-4d86-936a-1f10336654df req-0b5c849b-c237-4352-9bb0-c2617aa9164b tempest-TestShelveInstance-1011857455 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Oct 01 22:09:41.302784 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_concurrency.processutils [req-468ffc46-caf4-4d86-936a-1f10336654df req-0b5c849b-c237-4352-9bb0-c2617aa9164b tempest-TestShelveInstance-1011857455 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.729s {{(pid=133391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Oct 01 22:09:41.303553 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-468ffc46-caf4-4d86-936a-1f10336654df req-0b5c849b-c237-4352-9bb0-c2617aa9164b tempest-TestShelveInstance-1011857455 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-223aaf60-9290-4c78-830a-bd61587db2a6', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'b6fe18b3-7cfc-4817-97e0-9c52d2ce9a67', 'volume_id': '223aaf60-9290-4c78-830a-bd61587db2a6', 'discard': True}} {{(pid=133391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1783}} Oct 01 22:09:41.328402 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-468ffc46-caf4-4d86-936a-1f10336654df req-0b5c849b-c237-4352-9bb0-c2617aa9164b tempest-TestShelveInstance-1011857455 None] Connection info returned from driver {'name': 'volumes/volume-223aaf60-9290-4c78-830a-bd61587db2a6', 'hosts': ['10.222.0.17'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '223aaf60-9290-4c78-830a-bd61587db2a6', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '31e67b62-d473-4768-9668-f2b246e59cf9'} {{(pid=133391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Oct 01 22:09:41.376685 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-468ffc46-caf4-4d86-936a-1f10336654df req-0b5c849b-c237-4352-9bb0-c2617aa9164b tempest-TestShelveInstance-1011857455 None] attachment_update completed successfully. Oct 01 22:09:46.027638 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=16,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:10:06.035703 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=13,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:10:16.045178 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:10:23.924766 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:10:23.925671 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:10:23.951890 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:10:23.974807 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:10:36.046091 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:10:36.048793 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:10:56.044887 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:12.257046 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:12.269408 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-01c154fd-1ec8-4e0c-ab67-991cd3f03b52 tempest-TestShelveInstance-1011857455 None] Terminate volume connection completed successfully. Oct 01 22:11:12.270131 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.manager [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-01c154fd-1ec8-4e0c-ab67-991cd3f03b52 tempest-TestShelveInstance-1011857455 None] Deleting attachment 31e67b62-d473-4768-9668-f2b246e59cf9. {{(pid=133391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Oct 01 22:11:12.391921 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] Acquiring lock "cinder-223aaf60-9290-4c78-830a-bd61587db2a6-delete_volume" by "delete_volume" {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Oct 01 22:11:12.401701 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] Lock "cinder-223aaf60-9290-4c78-830a-bd61587db2a6-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Oct 01 22:11:12.435146 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:11:12.509702 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] volume has no backup snaps {{(pid=133391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Oct 01 22:11:12.517025 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] deleting rbd volume volume-223aaf60-9290-4c78-830a-bd61587db2a6 {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Oct 01 22:11:12.727872 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] volume is a clone so cleaning references {{(pid=133391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Oct 01 22:11:12.756653 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] deleting parent snapshot volume-223aaf60-9290-4c78-830a-bd61587db2a6.clone_snap {{(pid=133391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Oct 01 22:11:13.350215 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.quota [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] Created reservations ['cedf770e-445c-4c80-a7bd-42057c593f7b', '36d6d984-2a9e-4098-8ca3-d0af7892b74b', '121eb3d7-0cdf-4995-82e0-c0d7c17ae20f', '9723d244-b34b-4803-8208-690e68d1d25e'] {{(pid=133391) reserve /opt/stack/cinder/cinder/quota.py:755}} Oct 01 22:11:13.397860 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:11:13.421329 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:11:13.445674 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:11:13.451540 ncdv-master-1 cinder-volume[133391]: INFO cinder.volume.manager [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] Deleted volume successfully. Oct 01 22:11:13.463473 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.coordination [req-8af4c296-2a4f-4cf3-81b6-47f27d868ac8 req-e2e4baef-e5dd-407d-9dbd-695671bed89f tempest-TestShelveInstance-1011857455 None] Lock "cinder-223aaf60-9290-4c78-830a-bd61587db2a6-delete_volume" released by "delete_volume" :: held 1.062s {{(pid=133391) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Oct 01 22:11:23.921487 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:11:23.922712 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:11:23.967129 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:11:24.051345 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:11:26.060037 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=19,cinder:UPDATE=9,cinder:INSERT=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:46.061351 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:11:46.062980 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=19,cinder:INSERT=2,cinder:UPDATE=10,cinder:DELETE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:12:23.923540 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:12:23.924275 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:12:23.947479 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:12:23.970504 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:12:26.063513 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:12:26.066882 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:12:56.077695 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:12:56.079446 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:13:23.925418 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:13:23.926076 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:13:23.949736 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:13:23.973706 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:13:26.089673 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:13:26.091215 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:13:46.097297 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:13:56.100553 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:14:16.104575 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:14:16.105201 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:14:23.928306 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:14:23.928982 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:14:23.952612 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:14:23.975895 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:14:36.105728 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:14:56.109785 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=9,cinder:UPDATE=4 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:15:06.111602 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:15:23.931338 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:15:23.932000 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:15:23.955469 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:15:23.979288 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:15:26.119264 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:15:36.123952 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:15:56.128869 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:16:06.141158 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:16:23.933922 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:16:23.934613 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:16:23.958118 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:16:23.982049 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:16:26.143580 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:16:46.144719 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:16:56.154425 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:17:16.156509 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:17:26.165428 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:17:46.166093 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:18:06.172591 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:18:16.177789 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:18:23.939484 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:18:23.940139 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:18:23.965649 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:18:23.993521 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:18:26.183989 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:18:36.190007 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:18:56.190079 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:19:06.200413 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:19:23.941683 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:19:23.942332 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:19:23.965671 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:19:23.990940 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:19:26.205645 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:19:36.209512 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:19:56.215851 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:20:06.220832 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:20:23.945704 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:20:23.946380 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:20:23.969864 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:20:23.992975 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:20:26.226545 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:20:36.230449 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:20:56.231128 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:20:56.232829 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:UPDATE=2,cinder:SELECT=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:21:16.242841 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:21:23.948300 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:21:23.948975 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:21:23.976092 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:21:24.003467 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:21:26.244393 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:21:46.252136 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:21:56.259913 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:22:16.267184 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:22:23.949255 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:22:23.949930 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:22:23.973921 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:22:23.997687 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:22:26.269811 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:22:36.273354 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:22:56.280027 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:23:06.283634 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:23:23.952847 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:23:23.953523 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:23:23.976294 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:23:24.000893 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:23:36.281880 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:23:46.287849 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:23:56.296893 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:24:16.294322 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=1 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:24:16.297383 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 01 22:24:23.955469 ncdv-master-1 cinder-volume[133391]: DEBUG oslo_service.periodic_task [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Oct 01 22:24:23.956116 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:24:23.979866 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.volume.drivers.rbd [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Oct 01 22:24:24.003139 ncdv-master-1 cinder-volume[133391]: DEBUG cinder.manager [None req-c4a1dc2a-11a2-40f9-9704-edc14f4fba95 None None] Notifying Schedulers of capabilities ... {{(pid=133391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Oct 01 22:24:36.307558 ncdv-master-1 cinder-volume[133391]: DEBUG dbcounter [-] [133391] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=133391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}