-- Logs begin at Sat 2022-09-17 20:01:16 UTC, end at Sat 2022-09-17 22:18:51 UTC. -- Sep 17 20:40:56.153507 ncdv-master-1 systemd[1]: Started Devstack devstack@c-vol.service. Sep 17 20:40:59.494441 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "singleton_lock" {{(pid=133242) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 17 20:40:59.495314 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=133242) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 17 20:40:59.495314 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=133242) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 17 20:40:59.901967 ncdv-master-1 cinder-volume[133242]: INFO dbcounter [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Registered counter for database cinder Sep 17 20:40:59.923206 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_db.sqlalchemy.engines [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133242) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 20:40:59.930079 ncdv-master-1 cinder-volume[133242]: DEBUG dbcounter [-] [133242] Writer thread running {{(pid=133242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 20:41:00.222164 ncdv-master-1 cinder-volume[133242]: INFO cinder.rpc [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version. Sep 17 20:41:00.227103 ncdv-master-1 cinder-volume[133242]: INFO cinder.rpc [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version. Sep 17 20:41:00.234772 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_db.sqlalchemy.engines [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133242) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 20:41:00.301228 ncdv-master-1 cinder-volume[133242]: INFO cinder.volume.manager [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Determined volume DB was empty at startup. Sep 17 20:41:00.301882 ncdv-master-1 cinder-volume[133242]: DEBUG cinder.volume.manager [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Cinder Volume DB check: vol_db_empty=True {{(pid=133242) __init__ /opt/stack/cinder/cinder/volume/manager.py:290}} Sep 17 20:41:00.310625 ncdv-master-1 cinder-volume[133242]: INFO cinder.volume.manager [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Service not found for updating active_backend_id, assuming default for driver init. Sep 17 20:41:00.311404 ncdv-master-1 cinder-volume[133242]: WARNING stevedore.named [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager Sep 17 20:41:00.311847 ncdv-master-1 cinder-volume[133242]: WARNING castellan.key_manager [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' Sep 17 20:41:00.312967 ncdv-master-1 cinder-volume[133242]: WARNING cinder.keymgr.conf_key_mgr [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] This key manager is insecure and is not recommended for production deployments: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' Sep 17 20:41:00.331328 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_db.sqlalchemy.engines [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133242) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 20:41:00.335868 ncdv-master-1 cinder-volume[133242]: INFO cinder.rpc [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version. Sep 17 20:41:00.340152 ncdv-master-1 cinder-volume[133242]: INFO cinder.rpc [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Automatically selected cinder-volume RPC version 3.18 as minimum service version. Sep 17 20:41:00.340972 ncdv-master-1 cinder-volume[133242]: WARNING stevedore.named [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager Sep 17 20:41:00.341376 ncdv-master-1 cinder-volume[133242]: WARNING castellan.key_manager [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' Sep 17 20:41:00.343128 ncdv-master-1 cinder-volume[133242]: INFO cinder.volume.manager [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Image-volume cache enabled for host ncdv-master-1@ceph. Sep 17 20:41:00.348827 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_db.sqlalchemy.engines [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133242) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 20:41:00.368142 ncdv-master-1 cinder-volume[133242]: INFO oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Starting 1 workers Sep 17 20:41:00.373267 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Started child 133409 {{(pid=133242) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}} Sep 17 20:41:00.374270 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Full set of CONF: {{(pid=133242) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}} Sep 17 20:41:00.374641 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ******************************************************************************** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 17 20:41:00.374947 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] Configuration options gathered from: {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 17 20:41:00.375293 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 17 20:41:00.376044 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 17 20:41:00.376327 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ================================================================================ {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 17 20:41:00.377072 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] allow_availability_zone_fallback = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.377442 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] allow_compression_on_image_upload = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.377943 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] allowed_direct_url_schemes = [] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.378324 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] api_rate_limit = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.378832 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] auth_strategy = keystone {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.379444 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] az_cache_duration = 3600 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.379978 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_availability_zone = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.380365 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_host = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.380741 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_name = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.381124 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_stats_polling_interval = 60 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.381510 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backup_api_class = cinder.backup.api.API {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.381890 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backup_manager = cinder.backup.manager.BackupManager {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.382282 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backup_name_template = backup-%s {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.382543 ncdv-master-1 cinder-volume[133409]: INFO cinder.service [-] Starting cinder-volume node (version 20.1.0) Sep 17 20:41:00.383076 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backup_use_same_host = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.383076 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backup_use_temp_snapshot = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.383384 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] chap_password = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.383757 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] chap_username = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.384127 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.384695 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cinder_internal_tenant_project_id = 58e0a10f1c5b4b76b9d08df11fb7a377 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.385204 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cinder_internal_tenant_user_id = d45ab41902864586811c81948540fdac {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.385882 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cloned_volume_same_az = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.386383 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cluster = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.386890 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] compression_format = gzip {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.387441 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] compute_api_class = cinder.compute.nova.API {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.388009 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] config_dir = [] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.388451 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] config_file = ['/etc/cinder/cinder.conf'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.388972 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] config_source = [] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.389500 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] consistencygroup_api_class = cinder.consistencygroup.api.API {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.390045 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] control_exchange = openstack {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.390582 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] debug = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.391091 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] default_availability_zone = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.391581 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] default_group_type = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.392381 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.392926 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] default_volume_type = ceph {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.393421 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] driver_client_cert = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.393924 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] driver_client_cert_key = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.394520 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] driver_data_namespace = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.395035 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] driver_ssl_cert_path = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.395692 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] driver_ssl_cert_verify = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.396326 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] driver_use_ssl = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.396838 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] enable_force_upload = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.397305 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] enable_new_services = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.397809 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] enable_unsupported_driver = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.398347 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] enabled_backends = ['ceph'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.398828 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] enforce_multipath_for_image_xfer = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.399314 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] filter_function = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.399823 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] glance_api_insecure = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.400331 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] glance_api_servers = ['http://10.222.0.26/image'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.400809 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] glance_api_ssl_compression = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.401302 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] glance_ca_certificates_file = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.401850 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] glance_catalog_info = image:glance:publicURL {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.402350 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] glance_certfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.402974 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] glance_core_properties = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.403467 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] glance_keyfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.403957 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] glance_num_retries = 3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.404431 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] glance_request_timeout = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.404916 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] goodness_function = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.405836 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] graceful_shutdown_timeout = 5 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.406464 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] group_api_class = cinder.group.api.API {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.406901 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] host = ncdv-master-1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.407411 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] iet_conf = /etc/iet/ietd.conf {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.407947 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] image_compress_on_upload = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.409083 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] image_conversion_address_space_limit = 1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.409655 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] image_conversion_cpu_limit = 60 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.410366 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] image_conversion_dir = /opt/stack/data/cinder/conversion {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.410903 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] image_conversion_disable = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.411399 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] image_upload_use_cinder_backend = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.411893 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] image_upload_use_internal_tenant = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.412374 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] image_volume_cache_enabled = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.412865 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] image_volume_cache_max_count = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.413379 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] image_volume_cache_max_size_gb = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.413901 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] init_host_max_objects_retrieval = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.414057 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Starting volume driver RBDDriver (1.3.0) Sep 17 20:41:00.414550 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] instance_format = [instance: %(uuid)s] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.415063 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.415609 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] iscsi_iotype = fileio {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.415716 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:41:00.416336 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] iscsi_secondary_ip_addresses = [] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.416866 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] iscsi_target_flags = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.417392 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] iscsi_write_cache = on {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.417933 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] iser_helper = tgtadm {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.418670 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] iser_ip_address = 10.222.0.26 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.419222 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] iser_port = 3260 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.419832 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.420334 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] log_config_append = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.420850 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.421347 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] log_dir = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.421961 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] log_file = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.422404 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] log_options = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.422931 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] log_rotate_interval = 1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.423473 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] log_rotate_interval_type = days {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.423993 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] log_rotation_type = none {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.424424 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.424867 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.425374 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.425983 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.426418 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.427054 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] max_age = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.427562 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] max_logfile_count = 30 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.428061 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] max_logfile_size_mb = 200 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.428589 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] max_over_subscription_ratio = 20.0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.429125 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] message_reap_interval = 86400 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.429659 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] message_ttl = 2592000 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.430181 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] migration_create_volume_timeout_secs = 300 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.430690 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] monkey_patch = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.431187 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] monkey_patch_modules = [] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.431625 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] my_ip = 10.222.0.26 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.432141 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] no_snapshot_gb_quota = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.432491 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] num_iser_scan_tries = 3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.432899 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] num_shell_tries = 3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.433273 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] num_volume_device_scan_tries = 3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.433642 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] nvmet_ns_id = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.433986 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] nvmet_port_id = 1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.434439 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] osapi_max_limit = 1000 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.434726 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] osapi_volume_ext_list = [] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.435151 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] osapi_volume_extension = ['cinder.api.contrib.standard_extensions'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.435509 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] osapi_volume_listen = 0.0.0.0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.435873 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] osapi_volume_listen_port = 8776 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.436217 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] osapi_volume_use_ssl = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.436565 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] osapi_volume_workers = 2 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.437151 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] per_volume_size_limit = -1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.437676 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] periodic_fuzzy_delay = 60 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.438199 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] periodic_interval = 60 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.439181 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] project_id_regex = [0-9a-f\-]+ {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.439967 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] public_endpoint = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.440460 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] publish_errors = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.440986 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] quota_backup_gigabytes = 1000 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.441591 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] quota_backups = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.441945 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] quota_consistencygroups = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.442327 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] quota_driver = cinder.quota.DbQuotaDriver {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.442671 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] quota_gigabytes = 1000 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.443021 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] quota_groups = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.443384 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] quota_snapshots = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.443753 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] quota_volumes = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.444092 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] rate_limit_burst = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.444831 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] rate_limit_except_level = CRITICAL {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.445329 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] rate_limit_interval = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.445935 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] reinit_driver_count = 3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.446457 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] replication_device = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.446806 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] report_discard_supported = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.448537 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] report_interval = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.449027 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] reservation_clean_interval = 86400 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.449321 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] reservation_expire = 86400 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.449715 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] reserved_percentage = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.450095 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] resource_query_filters_file = /etc/cinder/resource_filters.json {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.450449 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] rootwrap_config = /etc/cinder/rootwrap.conf {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.450853 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] rpc_response_timeout = 120 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.451217 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] run_external_periodic_tasks = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.451570 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] scheduler_manager = cinder.scheduler.manager.SchedulerManager {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.451947 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] scst_target_driver = iscsi {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.452294 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] scst_target_iqn_name = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.452580 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] service_down_time = 60 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.453338 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] snapshot_name_template = snapshot-%s {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.453338 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] snapshot_same_host = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.453609 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] split_loggers = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.454879 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] state_path = /opt/stack/data/cinder {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.456196 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] storage_availability_zone = nova {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.456704 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] storage_protocol = iSCSI {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.457430 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] syslog_log_facility = LOG_USER {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.457962 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] target_helper = lioadm {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.458559 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] target_ip_address = 10.222.0.26 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.459080 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] target_port = 3260 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.459600 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] target_prefix = iqn.2010-10.org.openstack: {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.460129 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] target_protocol = iscsi {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.460911 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] trace_flags = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.465535 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] transfer_api_class = cinder.transfer.api.API {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.466233 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] transport_url = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.466652 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] until_refresh = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.467012 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] use_chap_auth = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.467508 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] use_default_quota_class = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.468052 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] use_eventlog = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.468418 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] use_journal = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.468783 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] use_json = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.469135 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] use_multipath_for_image_xfer = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.469506 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] use_stderr = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.470317 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] use_syslog = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.470870 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] verify_glance_signatures = enabled {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.471521 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_api_class = cinder.volume.api.API {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.472044 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_backend_name = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.472636 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_clear = zero {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.473135 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_clear_ionice = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.473559 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:41:00.473825 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_clear_size = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.474349 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.474866 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_copy_bps_limit = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.475379 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_dd_blocksize = 1M {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.475893 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_manager = cinder.volume.manager.VolumeManager {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.476402 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_name_template = volume-%s {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.476912 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_service_inithost_offload = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.477424 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volume_usage_audit_period = month {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.478074 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] volumes_dir = /opt/stack/data/cinder/volumes {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.478561 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] watch_log_file = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.479045 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] zoning_mode = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 17 20:41:00.479663 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] os_brick.lock_path = /opt/stack/data/cinder {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.480227 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.480786 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.481344 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_concurrency.disable_process_locking = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.481915 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_concurrency.lock_path = /opt/stack/data/cinder {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.482480 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.483005 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.483539 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.484052 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.484544 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.485232 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.admin_password = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.485776 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.486290 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.admin_token = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.486637 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.admin_user = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.487025 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.auth_admin_prefix = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.487406 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.487777 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.auth_port = 35357 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.488206 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.auth_protocol = https {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.488672 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.auth_section = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.489057 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.auth_type = password {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.489426 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.auth_uri = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.489928 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.auth_version = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.490262 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.cache = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.490621 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.490976 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.certfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.491474 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.delay_auth_decision = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.491713 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.492050 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.http_connect_timeout = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.492414 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.492753 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.identity_uri = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.493121 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.include_service_catalog = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.493490 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.insecure = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.493848 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.interface = public {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.494187 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.keyfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.494545 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.494894 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.495450 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.495586 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.495932 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.496276 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.496652 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.497018 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.497413 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.497939 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.region_name = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.498313 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.498674 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.service_token_roles_required = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.499012 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.service_type = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.499446 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.token_cache_time = 300 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.500195 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.501009 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.backend = dogpile.cache.null {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.501385 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.backend_argument = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.501816 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.config_prefix = cache.oslo {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.502275 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.dead_timeout = 60.0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.502642 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.debug_cache_backend = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.503001 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.enable_retry_client = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.503509 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.enable_socket_keepalive = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.504040 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.enabled = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.504557 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.expiration_time = 600 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.505092 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.hashclient_retry_attempts = 2 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.505426 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.hashclient_retry_delay = 1.0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.505905 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.memcache_dead_retry = 300 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.506277 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.memcache_password = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.506644 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.506989 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.507342 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.memcache_pool_maxsize = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.507711 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.508052 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.memcache_sasl_enabled = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.508419 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.memcache_servers = ['localhost:11211'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.508803 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.memcache_socket_timeout = 1.0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.509153 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.memcache_username = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.509710 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.proxies = [] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.510066 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.retry_attempts = 2 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.510402 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.retry_delay = 0.0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.510749 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.socket_keepalive_count = 1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.511090 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.socket_keepalive_idle = 1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.511431 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.socket_keepalive_interval = 1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.511763 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.tls_allowed_ciphers = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.512124 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.tls_cafile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.512457 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.tls_certfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.512800 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.tls_enabled = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.513141 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cache.tls_keyfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.513773 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] coordination.backend_url = etcd3+http://10.222.0.26:2379 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.515267 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.backend = sqlalchemy {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.515592 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.connection = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.516001 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.connection_debug = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.516368 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.connection_parameters = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.516748 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.connection_recycle_time = 3600 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.517112 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.connection_trace = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.517481 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.db_inc_retry_interval = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.517825 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.db_max_retries = 20 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.518172 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.db_max_retry_interval = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.518519 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.db_retry_interval = 1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.518874 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.max_overflow = 50 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.519219 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.max_pool_size = 5 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.519679 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.max_retries = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.520022 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.mysql_enable_ndb = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.520379 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.520732 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.pool_timeout = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.521114 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.retry_interval = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.521480 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.slave_connection = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.522402 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.sqlite_synchronous = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.522776 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] database.use_db_reconnect = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.523151 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_policy.enforce_new_defaults = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.523515 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_policy.enforce_scope = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.523912 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_policy.policy_default_rule = default {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.524302 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.524698 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_policy.policy_file = policy.yaml {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.525100 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.525447 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.525831 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.526204 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.526566 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.526985 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] profiler.connection_string = messaging:// {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.527275 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] profiler.enabled = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.527629 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] profiler.es_doc_type = notification {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.527993 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] profiler.es_scroll_size = 10000 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.528351 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] profiler.es_scroll_time = 2m {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.528715 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] profiler.filter_error_trace = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.529064 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] profiler.hmac_keys = SECRET_KEY {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.529417 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] profiler.sentinel_service_name = mymaster {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.530077 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] profiler.socket_timeout = 0.1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.530801 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] profiler.trace_sqlalchemy = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.531200 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] privsep_osbrick.capabilities = [21] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.531718 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] privsep_osbrick.group = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.532218 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] privsep_osbrick.helper_command = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.534084 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] privsep_osbrick.logger_name = oslo_privsep.daemon {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.534450 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.534820 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] privsep_osbrick.user = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.535249 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cinder_sys_admin.capabilities = [0, 1, 2, 3, 12, 21] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.535591 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cinder_sys_admin.group = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.535931 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cinder_sys_admin.helper_command = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.536283 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cinder_sys_admin.logger_name = oslo_privsep.daemon {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.536628 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.536963 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] cinder_sys_admin.user = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.537340 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] service_user.auth_section = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.537981 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] service_user.auth_type = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.538489 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] service_user.cafile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.538843 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] service_user.certfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.539208 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] service_user.collect_timing = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.539551 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] service_user.insecure = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.540028 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] service_user.keyfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.540382 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] service_user.send_service_user_token = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.540748 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] service_user.split_loggers = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.541103 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] service_user.timeout = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.541560 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.backend_availability_zone = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.541907 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.backend_native_threads_pool_size = 20 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.542431 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.backup_use_temp_snapshot = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.542944 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.chap_password = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.543472 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.chap_username = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.543830 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.544195 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.544542 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.544897 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.driver_client_cert = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.545235 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.driver_client_cert_key = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.545603 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.driver_data_namespace = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.545948 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.546304 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.546634 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.driver_use_ssl = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.546977 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.enable_deferred_deletion = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.547327 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.enable_unsupported_driver = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.547668 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.548025 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.extra_capabilities = {} {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.548360 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.filter_function = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.548696 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.goodness_function = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.549052 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.iet_conf = /etc/iet/ietd.conf {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.549391 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.549753 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.550098 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.550445 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.550870 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.551284 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.iscsi_iotype = fileio {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.551604 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.551971 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.iscsi_target_flags = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.552315 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.iscsi_write_cache = on {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.552666 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.iser_helper = tgtadm {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.553206 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.iser_ip_address = 10.222.0.26 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.553628 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.iser_port = 3260 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.553992 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.554354 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.554889 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.555258 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.num_shell_tries = 3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.555605 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.555956 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.nvmet_ns_id = 10 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.556308 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.nvmet_port_id = 1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.556661 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.557003 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rados_connection_interval = 5 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.557350 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rados_connection_retries = 3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.557814 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rbd_ceph_conf = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.558118 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.558466 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.558824 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.559169 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.559529 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rbd_pool = rbd {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.559865 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rbd_secret_uuid = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.560237 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.560577 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.rbd_user = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.560943 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.561301 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.replication_device = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.561661 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.report_discard_supported = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.562008 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.562356 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.reserved_percentage = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.562710 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.scst_target_driver = iscsi {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.563051 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.scst_target_iqn_name = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.563428 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.storage_protocol = iSCSI {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.563770 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.suppress_requests_ssl_warnings = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.564125 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.target_helper = tgtadm {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.564599 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.target_ip_address = 10.222.0.26 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.564943 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.target_port = 3260 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.565316 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.565710 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.target_protocol = iscsi {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.566047 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.trace_flags = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.566484 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.unique_fqdn_network = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.566889 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.use_chap_auth = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.567256 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.567602 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.volume_backend_name = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.567962 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.volume_clear = zero {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.568297 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.volume_clear_ionice = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.568639 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.volume_clear_size = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.569022 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.569382 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.569757 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.570117 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.570577 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] backend_defaults.volumes_dir = /opt/stack/data/cinder/volumes {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.570952 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_reports.file_event_handler = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.571321 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.571711 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_reports.log_dir = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.572056 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.backend_availability_zone = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.572390 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.backend_host = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.572731 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.backend_native_threads_pool_size = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.573118 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.backup_use_temp_snapshot = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.573402 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.chap_password = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.573767 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.chap_username = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.574106 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.chiscsi_conf = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.574459 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.deferred_deletion_delay = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.574793 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.deferred_deletion_purge_interval = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.575128 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.driver_client_cert = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.575482 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.driver_client_cert_key = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.575828 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.driver_data_namespace = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.576186 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.driver_ssl_cert_path = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.576522 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.driver_ssl_cert_verify = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.576903 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.driver_use_ssl = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.577193 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.enable_deferred_deletion = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.577681 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.enable_unsupported_driver = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.577990 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.enforce_multipath_for_image_xfer = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.578329 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.extra_capabilities = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.578662 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.filter_function = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.578996 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.goodness_function = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.579326 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.iet_conf = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.579661 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.image_upload_use_cinder_backend = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.579990 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.image_upload_use_internal_tenant = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.580349 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.image_volume_cache_enabled = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.580689 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.image_volume_cache_max_count = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.581024 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.image_volume_cache_max_size_gb = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.581357 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.iscsi_iotype = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.581716 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.iscsi_secondary_ip_addresses = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.582107 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.iscsi_target_flags = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.582434 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.iscsi_write_cache = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.583636 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.iser_helper = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.583636 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.iser_ip_address = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.583636 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.iser_port = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.584038 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.iser_target_prefix = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.584038 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.max_over_subscription_ratio = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.584390 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.num_iser_scan_tries = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.584706 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.num_shell_tries = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.585065 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.num_volume_device_scan_tries = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.585405 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.nvmet_ns_id = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.585835 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.nvmet_port_id = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.586170 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rados_connect_timeout = -1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.586568 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rados_connection_interval = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.587046 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rados_connection_retries = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.587337 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rbd_ceph_conf = /etc/ceph/ceph.conf {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.587724 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rbd_cluster_name = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.588060 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rbd_exclusive_cinder_pool = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.588713 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rbd_flatten_volume_from_snapshot = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.589260 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rbd_max_clone_depth = 5 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.589954 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rbd_pool = volumes {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.590138 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rbd_secret_uuid = 4391210e-4b15-4c1d-b970-cae3efa14448 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.590359 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rbd_store_chunk_size = 4 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.590774 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.rbd_user = cinder {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.591109 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.replication_connect_timeout = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.591451 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.replication_device = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.591785 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.report_discard_supported = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.592124 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.report_dynamic_total_capacity = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.592639 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.reserved_percentage = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.592976 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.scst_target_driver = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.593335 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.scst_target_iqn_name = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.593694 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.storage_protocol = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.594030 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.suppress_requests_ssl_warnings = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.594441 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.target_helper = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.594773 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.target_ip_address = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.595100 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.target_port = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.595359 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.target_prefix = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.595692 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.target_protocol = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.596023 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.trace_flags = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.596353 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.unique_fqdn_network = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.596690 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.use_chap_auth = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.597031 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.use_multipath_for_image_xfer = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.597382 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.volume_backend_name = ceph {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.597787 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.volume_clear = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.598099 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.volume_clear_ionice = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.598432 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.volume_clear_size = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.598764 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.volume_copy_blkio_cgroup_name = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.599097 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.volume_copy_bps_limit = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.599428 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.volume_dd_blocksize = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.599822 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.volume_driver = cinder.volume.drivers.rbd.RBDDriver {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.600136 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ceph.volumes_dir = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.600544 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.600882 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.601247 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.601737 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.602195 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.602451 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.602816 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.603182 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.603520 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.603872 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.604244 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.604593 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.604965 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.605315 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.605695 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.606230 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.606576 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.606919 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.607324 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.607601 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.607939 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.608286 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.608645 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.609013 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.609386 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.609772 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.ssl = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.610147 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.610505 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.610842 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.611225 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.611579 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_rabbit.ssl_version = {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.611910 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_notifications.driver = [] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.612314 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_notifications.retry = -1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.612668 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.613009 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] oslo_messaging_notifications.transport_url = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.613453 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] key_manager.backend = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.613767 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] key_manager.fixed_key = **** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.614191 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.614526 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.barbican_api_version = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.614886 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.barbican_endpoint = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.615271 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.barbican_endpoint_type = public {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.615624 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.barbican_region_name = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.615956 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.cafile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.616287 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.certfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.616637 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.collect_timing = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.616974 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.insecure = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.617327 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.keyfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.617768 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.number_of_retries = 60 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.618084 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.retry_delay = 1 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.618418 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.send_service_user_token = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.618756 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.split_loggers = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.619091 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.timeout = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.619424 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.verify_ssl = True {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.619780 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican.verify_ssl_path = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.620159 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican_service_user.auth_section = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.620494 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican_service_user.auth_type = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.620877 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican_service_user.cafile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.621205 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican_service_user.certfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.621527 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican_service_user.collect_timing = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.622945 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican_service_user.insecure = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.623281 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican_service_user.keyfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.623617 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican_service_user.split_loggers = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.623948 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] barbican_service_user.timeout = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.624324 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.approle_role_id = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.624661 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.approle_secret_id = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.624991 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.cafile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.625322 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.certfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.625676 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.collect_timing = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.626009 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.insecure = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.626341 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.keyfile = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.626698 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.kv_mountpoint = secret {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.627049 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.kv_version = 2 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.627379 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.namespace = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.627773 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.root_token_id = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.628042 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.split_loggers = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.628371 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.ssl_ca_crt_file = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.628703 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.timeout = None {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.629073 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.use_ssl = False {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.629424 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 17 20:41:00.629746 ncdv-master-1 cinder-volume[133242]: DEBUG oslo_service.service [None req-fdb6c5c8-1154-4a54-9faf-4d3ae31ed026 None None] ******************************************************************************** {{(pid=133242) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 17 20:41:00.644027 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:41:00.681511 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.driver [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Driver hasn't implemented _init_vendor_properties() Sep 17 20:41:00.681781 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.driver [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Initialized capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.3.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:b7523680-622f-42b1-bdc5-bda0e7b4a981:cinder:volumes', 'backend_state': 'up', 'qos_support': True, 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=133409) init_capabilities /opt/stack/cinder/cinder/volume/driver.py:812}} Sep 17 20:41:00.700228 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_db.sqlalchemy.engines [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=133409) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 17 20:41:00.702665 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writer thread running {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 17 20:41:00.743741 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.db.sqlalchemy.api [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Building query based on filter {{(pid=133409) _process_snaps_filters /opt/stack/cinder/cinder/db/sqlalchemy/api.py:3885}} Sep 17 20:41:00.765433 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.image_utils [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Configuration image_conversion_dir is None or the path doesn't exist. {{(pid=133409) cleanup_temporary_file /opt/stack/cinder/cinder/image/image_utils.py:991}} Sep 17 20:41:00.771857 ncdv-master-1 cinder-volume[133409]: INFO cinder.keymgr.migration [None req-e9728146-0301-4380-a802-222319830621 None None] Not migrating encryption keys because the ConfKeyManager is still in use. Sep 17 20:41:00.779788 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:41:00.807108 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:41:00.836050 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:41:00.862890 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Driver initialization completed successfully. Sep 17 20:41:00.863744 ncdv-master-1 cinder-volume[133409]: INFO cinder.manager [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Initiating service 3 cleanup Sep 17 20:41:00.874869 ncdv-master-1 cinder-volume[133409]: INFO cinder.manager [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Service 3 cleanup completed. Sep 17 20:41:00.875351 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.service [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Creating RPC server for service cinder-volume {{(pid=133409) start /opt/stack/cinder/cinder/service.py:238}} Sep 17 20:41:00.883909 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.service [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=133409) start /opt/stack/cinder/cinder/service.py:244}} Sep 17 20:41:00.928835 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Initializing RPC dependent components of volume driver RBDDriver (1.3.0) Sep 17 20:41:00.930037 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:41:00.966150 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:41:01.012309 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-285e0a52-2a5c-49c7-93fe-163cae9ac17e None None] Driver post RPC initialization completed successfully. Sep 17 20:41:10.361690 ncdv-master-1 cinder-volume[133242]: DEBUG dbcounter [-] [133242] Writing DB stats cinder:SELECT=16,cinder:INSERT=1 {{(pid=133242) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:11.005980 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=24,cinder:INSERT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:11.008647 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:41.014927 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:41:51.008401 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=12,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:01.014181 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:42:01.014943 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:42:01.040891 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:01.044925 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:42:01.069117 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:42:21.022458 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:41.027140 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:42:51.033559 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:01.017248 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:43:01.017963 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:43:01.044263 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:01.046518 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:43:01.071596 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:43:21.049521 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:21.053646 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:43:41.056669 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:44:01.046200 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:44:21.055622 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:44:31.056099 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:44:51.053895 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:45:01.022576 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:45:01.023267 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:45:01.046898 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:45:01.069764 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:45:01.072981 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:45:31.074375 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:45:31.075534 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:45:51.080613 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:46:01.027333 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:46:01.028766 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:46:01.052162 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:46:01.075698 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:46:11.082038 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:46:21.097440 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:46:41.100394 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:47:01.032880 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:47:01.033994 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:47:01.060226 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:47:01.083272 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:47:01.095658 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:47:21.095287 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:47:41.092915 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:48:01.029386 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:48:01.030547 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:48:01.054059 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:48:01.077106 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:48:01.091559 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:48:21.089363 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:48:21.093499 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:48:51.102281 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:49:01.032841 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:49:01.033897 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:49:01.062115 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:49:01.086296 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:49:01.098060 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:49:11.100626 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:49:31.096006 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:49:51.097157 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:01.033911 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:50:01.034537 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:50:01.062625 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:50:01.085722 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:50:11.103787 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:11.108858 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:31.114017 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:50:41.122007 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:01.030941 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:51:01.031619 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:51:01.099707 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:51:01.159255 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:51:01.166928 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:01.167907 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:31.198581 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:51:31.201987 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:01.036432 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:52:01.037309 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:52:01.126357 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:52:01.290956 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:52:01.292490 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=1,cinder:SELECT=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:01.295665 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:41.306032 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=9,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:51.304346 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:52:56.531247 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 20:52:56.658726 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Flow 'volume_create_manager' (99509c2d-d0e2-4f1f-8702-5a7177616629) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:52:56.676696 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a2b218a7-e52e-406a-a2e1-b591368670bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:52:56.756170 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a2b218a7-e52e-406a-a2e1-b591368670bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T20:52:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1015086260',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d32359bf-ad08-4517-9759-3f42907f23c0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='26d60144477f4a5192d581350d678979',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T20:52:56Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T20:52:56Z,use_quota=True,user_id='6f7681de64c248268da4d78124f59b54',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:52:56.768893 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6655bbf6-6a43-4879-8a50-01a78ccdb48d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:52:56.778893 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6655bbf6-6a43-4879-8a50-01a78ccdb48d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:52:56.783975 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (9d863b6f-7869-46a9-b8a7-17819dda5029) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:52:56.943240 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (9d863b6f-7869-46a9-b8a7-17819dda5029) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'd32359bf-ad08-4517-9759-3f42907f23c0', 'volume_name': 'volume-d32359bf-ad08-4517-9759-3f42907f23c0', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:52:56.948901 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (18c55217-563f-46ff-9dfc-c5a6ef487e97) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:52:56.954063 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (18c55217-563f-46ff-9dfc-c5a6ef487e97) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:52:56.959693 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f63fbe6b-80a3-4c21-bfa6-044d4173c11b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:52:56.960976 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Volume d32359bf-ad08-4517-9759-3f42907f23c0: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-d32359bf-ad08-4517-9759-3f42907f23c0', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 20:52:56.961954 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Cloning d32359bf-ad08-4517-9759-3f42907f23c0 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 20:52:56.963070 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 20:52:56.979574 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 20:52:56.992986 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Preparing cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:889}} Sep 17 20:52:56.994558 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 20:53:01.039599 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:53:01.040315 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:53:01.098410 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:53:01.144651 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:53:14.760023 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.image_utils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Image fetch details: dest /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph, size 616.00 MB, duration 17.76 sec {{(pid=133409) fetch /opt/stack/cinder/cinder/image/image_utils.py:587}} Sep 17 20:53:14.760023 ncdv-master-1 cinder-volume[133409]: INFO cinder.image.image_utils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Image download 616.00 MB at 34.69 MB/s Sep 17 20:53:14.806541 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:53:14.973810 ncdv-master-1 sudo[145165]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph Sep 17 20:53:14.976214 ncdv-master-1 sudo[145165]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 17 20:53:17.302660 ncdv-master-1 sudo[145165]: pam_unix(sudo:session): session closed for user root Sep 17 20:53:17.304734 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph" returned: 0 in 2.498s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:53:17.312899 ncdv-master-1 cinder-volume[133409]: WARNING py.warnings [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] /usr/local/lib/python3.8/dist-packages/oslo_utils/imageutils.py:71: FutureWarning: The human format is deprecated and the format parameter will be removed. Use explicitly json instead in version 'xena' Sep 17 20:53:17.312899 ncdv-master-1 cinder-volume[133409]: debtcollector.deprecate( Sep 17 20:53:17.312899 ncdv-master-1 cinder-volume[133409]: Sep 17 20:53:17.313670 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.image_utils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Temporary image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 is fetched for user 6f7681de64c248268da4d78124f59b54. {{(pid=133409) fetch /opt/stack/cinder/cinder/image/image_utils.py:1124}} Sep 17 20:53:17.379223 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=12,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:17.398820 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:53:17.601274 ncdv-master-1 sudo[145187]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph Sep 17 20:53:17.603313 ncdv-master-1 sudo[145187]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 17 20:53:19.650973 ncdv-master-1 sudo[145187]: pam_unix(sudo:session): session closed for user root Sep 17 20:53:19.655140 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph" returned: 0 in 2.256s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:53:19.658897 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] creating volume 'volume-d32359bf-ad08-4517-9759-3f42907f23c0' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 20:53:19.662210 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:53:19.840886 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 20:53:19.841252 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 20:53:19.883810 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Attempting download of 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 ((None, None)) to volume d32359bf-ad08-4517-9759-3f42907f23c0. {{(pid=133409) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1190}} Sep 17 20:53:19.923023 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_qbg88ii6 {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:53:20.017490 ncdv-master-1 sudo[145342]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_qbg88ii6 Sep 17 20:53:20.019204 ncdv-master-1 sudo[145342]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 17 20:53:21.443302 ncdv-master-1 sudo[145342]: pam_unix(sudo:session): session closed for user root Sep 17 20:53:21.445750 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_qbg88ii6" returned: 0 in 1.522s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:53:21.446998 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:53:21.576618 ncdv-master-1 sudo[145352]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph Sep 17 20:53:21.578071 ncdv-master-1 sudo[145352]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 17 20:53:23.543749 ncdv-master-1 sudo[145352]: pam_unix(sudo:session): session closed for user root Sep 17 20:53:23.546614 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph" returned: 0 in 2.099s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:53:23.548074 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.image_utils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 was vhdx, converting to raw {{(pid=133409) fetch_to_volume_format /opt/stack/cinder/cinder/image/image_utils.py:825}} Sep 17 20:53:23.549389 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpd3gsvnh3 {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:53:23.573337 ncdv-master-1 sudo[145380]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpd3gsvnh3 Sep 17 20:53:23.576731 ncdv-master-1 sudo[145380]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 17 20:53:26.512882 ncdv-master-1 sudo[145380]: pam_unix(sudo:session): session closed for user root Sep 17 20:53:26.516467 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpd3gsvnh3" returned: 0 in 2.966s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:53:26.519648 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:53:26.654243 ncdv-master-1 sudo[145489]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph Sep 17 20:53:26.655046 ncdv-master-1 sudo[145489]: pam_unix(sudo:session): session opened for user root by (uid=0) Sep 17 20:53:28.235325 ncdv-master-1 sudo[145489]: pam_unix(sudo:session): session closed for user root Sep 17 20:53:28.238379 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph" returned: 0 in 1.718s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:53:28.239746 ncdv-master-1 cinder-volume[133409]: WARNING py.warnings [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] /usr/local/lib/python3.8/dist-packages/oslo_utils/imageutils.py:71: FutureWarning: The human format is deprecated and the format parameter will be removed. Use explicitly json instead in version 'xena' Sep 17 20:53:28.239746 ncdv-master-1 cinder-volume[133409]: debtcollector.deprecate( Sep 17 20:53:28.239746 ncdv-master-1 cinder-volume[133409]: Sep 17 20:53:28.241197 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.image_utils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Image conversion details: src /opt/stack/data/cinder/conversion/image_fetch_1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4_loovqquqncdv-master-1@ceph, size 700.00 MB, duration 2.97 sec, destination /opt/stack/data/cinder/conversion/tmpd3gsvnh3 {{(pid=133409) _convert_image /opt/stack/cinder/cinder/image/image_utils.py:419}} Sep 17 20:53:28.241802 ncdv-master-1 cinder-volume[133409]: INFO cinder.image.image_utils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Converted 700.00 MB image at 235.81 MB/s Sep 17 20:53:28.243528 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:53:28.353233 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 20:53:28.358995 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 20:53:28.390591 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] deleting rbd volume volume-d32359bf-ad08-4517-9759-3f42907f23c0 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 20:53:28.971206 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Running cmd (subprocess): rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmpd3gsvnh3 volume-d32359bf-ad08-4517-9759-3f42907f23c0 --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:53:29.866381 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=10,cinder:UPDATE=2,cinder:INSERT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:31.315377 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:53:36.580676 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] CMD "rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmpd3gsvnh3 volume-d32359bf-ad08-4517-9759-3f42907f23c0 --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 7.610s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:53:36.736752 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:53:36.873044 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 20:53:36.873655 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 20:53:36.874244 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Downloaded image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 ((None, None)) to volume d32359bf-ad08-4517-9759-3f42907f23c0 successfully. {{(pid=133409) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1232}} Sep 17 20:53:36.957198 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.image_utils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Temporary image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 for user 6f7681de64c248268da4d78124f59b54 is deleted. {{(pid=133409) fetch /opt/stack/cinder/cinder/image/image_utils.py:1128}} Sep 17 20:53:37.059959 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Deprecated: Default quota for resource: volumes_ceph is set by the default quota flag: quota_volumes_ceph, it is now deprecated. Please use the default quota class for default quota. Sep 17 20:53:37.061049 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Deprecated: Default quota for resource: gigabytes_ceph is set by the default quota flag: quota_gigabytes_ceph, it is now deprecated. Please use the default quota class for default quota. Sep 17 20:53:37.170737 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Created reservations ['a75a0c09-7116-4736-b9f9-4110f9bda72f', 'b82215fc-ed00-4d78-9b07-d2edc125ae54', 'c881f7a7-34d7-4ffb-8e16-8d5ec6eda84f', 'b4e724ee-96b5-4f73-ba2d-a0a38d0f4b7b'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 20:53:37.180960 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Creating image volume entry: {'metadata': {}, 'consistencygroup_id': None, 'volume_admin_metadata': [], 'group_id': None, 'display_description': None, 'provider_location': None, 'deleted': False, 'volume_glance_metadata': [{'key': 'signature_verified', 'value': 'False'}], 'volume_type_id': 'c8f1030c-5609-46eb-ad8a-9bbb096c5617', 'size': 1, 'replication_driver_data': None, 'provider_id': None, 'user_id': '6f7681de64c248268da4d78124f59b54', 'terminated_at': None, 'encryption_key_id': None, 'provider_geometry': None, 'replication_status': None, 'bootable': False, 'previous_status': None, 'cluster_name': None, 'deleted_at': None, 'display_name': 'image-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'provider_auth': None, 'ec2_id': None, 'created_at': datetime.datetime(2022, 9, 17, 20, 52, 56, tzinfo=datetime.timezone.utc), 'shared_targets': True, 'replication_extended_status': None, 'host': 'ncdv-master-1@ceph#ceph', 'project_id': '58e0a10f1c5b4b76b9d08df11fb7a377', 'volume_metadata': [], 'availability_zone': 'nova', 'launched_at': None, 'scheduled_at': datetime.datetime(2022, 9, 17, 20, 52, 56, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 52, 56, tzinfo=datetime.timezone.utc), 'multiattach': False, 'service_uuid': None, 'attach_status': 'detached', 'status': 'creating', 'source_volid': 'd32359bf-ad08-4517-9759-3f42907f23c0'}. {{(pid=133409) _clone_image_volume /opt/stack/cinder/cinder/volume/manager.py:1660}} Sep 17 20:53:37.294036 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Volume reschedule parameters: False retry: None {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 20:53:37.327092 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Flow 'volume_create_manager' (92667981-b4b2-4808-a1c7-4e0a8b7bf285) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:53:37.334550 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c576c9aa-f4be-4e40-8fc9-6486f4a697de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:53:37.356329 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c576c9aa-f4be-4e40-8fc9-6486f4a697de) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T20:52:56Z,deleted=False,deleted_at=None,display_description=None,display_name='image-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=79c922c8-a994-4aac-93a5-981c78ffc3d9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='58e0a10f1c5b4b76b9d08df11fb7a377',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T20:52:56Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=d32359bf-ad08-4517-9759-3f42907f23c0,status='creating',terminated_at=None,updated_at=2022-09-17T20:52:56Z,use_quota=True,user_id='6f7681de64c248268da4d78124f59b54',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:53:37.363693 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (077ecaee-3b7b-4b25-bc30-8ed588481b54) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:53:37.368598 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (077ecaee-3b7b-4b25-bc30-8ed588481b54) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:53:37.375371 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ac32591b-9a4d-4f8d-8aff-8d64457d457e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:53:37.393903 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ac32591b-9a4d-4f8d-8aff-8d64457d457e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'volume_name': 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9', 'volume_size': 1, 'source_volid': 'd32359bf-ad08-4517-9759-3f42907f23c0', 'source_volstatus': 'downloading'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:53:37.399805 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3af10eaf-a9ca-4095-8aed-569e1c1a417f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:53:37.405002 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3af10eaf-a9ca-4095-8aed-569e1c1a417f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:53:37.411369 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a150f837-77eb-4cbd-bb6d-94caa6f2dbc1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:53:37.414368 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Volume 79c922c8-a994-4aac-93a5-981c78ffc3d9: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9', 'volume_size': 1, 'source_volid': 'd32359bf-ad08-4517-9759-3f42907f23c0', 'source_volstatus': 'downloading'} Sep 17 20:53:37.432668 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:53:37.522854 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] creating snapshot='volume-79c922c8-a994-4aac-93a5-981c78ffc3d9.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 20:53:37.892544 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] cloning 'volume-d32359bf-ad08-4517-9759-3f42907f23c0@volume-79c922c8-a994-4aac-93a5-981c78ffc3d9.clone_snap' to 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 20:53:38.010307 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 20:53:38.011605 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 20:53:38.012187 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 20:53:38.043110 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 20:53:38.046998 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a150f837-77eb-4cbd-bb6d-94caa6f2dbc1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9', 'volume_size': 1, 'source_volid': 'd32359bf-ad08-4517-9759-3f42907f23c0', 'source_volstatus': 'downloading'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:53:38.059306 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6630f6df-5873-42fc-8bcd-a86500e373fe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:53:38.071213 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Volume volume-79c922c8-a994-4aac-93a5-981c78ffc3d9 (79c922c8-a994-4aac-93a5-981c78ffc3d9): created successfully Sep 17 20:53:38.075730 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6630f6df-5873-42fc-8bcd-a86500e373fe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:53:38.082875 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Flow 'volume_create_manager' (92667981-b4b2-4808-a1c7-4e0a8b7bf285) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:53:38.095270 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Created volume successfully. Sep 17 20:53:38.191191 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Creating new image-volume cache entry for image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on ncdv-master-1@ceph#ceph {{(pid=133409) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:104}} Sep 17 20:53:38.202237 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] New image-volume cache entry created: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 20, 53, 38, 195944)}. {{(pid=133409) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:129}} Sep 17 20:53:38.209326 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 41.230s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 20:53:38.210133 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Marking volume d32359bf-ad08-4517-9759-3f42907f23c0 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 20:53:38.224544 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to d32359bf-ad08-4517-9759-3f42907f23c0. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 20:53:38.236783 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Creating volume glance metadata for volume d32359bf-ad08-4517-9759-3f42907f23c0 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 20:53:38.341011 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f63fbe6b-80a3-4c21-bfa6-044d4173c11b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d32359bf-ad08-4517-9759-3f42907f23c0', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:53:38.354914 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (08928e8c-6ee4-43a2-b244-d9ac3e02221b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:53:38.389653 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0 (d32359bf-ad08-4517-9759-3f42907f23c0): created successfully Sep 17 20:53:38.394912 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (08928e8c-6ee4-43a2-b244-d9ac3e02221b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:53:38.405534 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Flow 'volume_create_manager' (99509c2d-d0e2-4f1f-8702-5a7177616629) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:53:38.420410 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2f22b450-ff8c-4baf-ab0a-e02776c4ac21 tempest-ServersTestBootFromVolume-594505527 None] Created volume successfully. Sep 17 20:53:44.419919 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-6bcfa418-21f2-4fef-9638-47900be31fd1 req-16a9abaf-0d9f-4d9d-9e86-eae174987eec tempest-ServersTestBootFromVolume-594505527 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:53:45.395602 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-6bcfa418-21f2-4fef-9638-47900be31fd1 req-16a9abaf-0d9f-4d9d-9e86-eae174987eec tempest-ServersTestBootFromVolume-594505527 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.975s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:53:45.396554 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-6bcfa418-21f2-4fef-9638-47900be31fd1 req-16a9abaf-0d9f-4d9d-9e86-eae174987eec tempest-ServersTestBootFromVolume-594505527 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-d32359bf-ad08-4517-9759-3f42907f23c0', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'd32359bf-ad08-4517-9759-3f42907f23c0', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:53:45.511667 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-6bcfa418-21f2-4fef-9638-47900be31fd1 req-16a9abaf-0d9f-4d9d-9e86-eae174987eec tempest-ServersTestBootFromVolume-594505527 None] Connection info returned from driver {'name': 'volumes/volume-d32359bf-ad08-4517-9759-3f42907f23c0', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'd32359bf-ad08-4517-9759-3f42907f23c0', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '57aef2c2-1c7b-4eda-b1fd-b8b2328bc398'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:53:45.704625 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-6bcfa418-21f2-4fef-9638-47900be31fd1 req-16a9abaf-0d9f-4d9d-9e86-eae174987eec tempest-ServersTestBootFromVolume-594505527 None] attachment_update completed successfully. Sep 17 20:54:01.035675 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:54:01.037047 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:54:01.163073 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:54:01.229919 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:54:01.308844 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=52,cinder:UPDATE=10,cinder:INSERT=14,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:11.322557 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=50,cinder:UPDATE=16,cinder:INSERT=10,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:31.320604 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:54:41.319019 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:00.111262 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 20:55:00.145346 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Flow 'volume_create_manager' (a1b1d973-cfac-4294-ba38-f9241c7c4bcb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:55:00.153502 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (656cc8a7-76b7-420a-bbe8-341cdfeb2303) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:00.174389 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (656cc8a7-76b7-420a-bbe8-341cdfeb2303) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T20:55:00Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveAutoBlockMigrationV225Test-volume-203155946',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d2a1c319-7ed9-4fb7-af62-52e9373195bf,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='7dc8bc81c2d74bae9612f7c9807350a7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T20:55:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T20:55:00Z,use_quota=True,user_id='f1134231b4a545fdb136a5204cb4f289',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:00.180747 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (aabc2086-1362-41de-a722-10a7a1440766) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:00.185661 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (aabc2086-1362-41de-a722-10a7a1440766) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:00.192541 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d3922bd9-e0af-4e38-9427-6c3da24a2db6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:00.197486 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d3922bd9-e0af-4e38-9427-6c3da24a2db6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd2a1c319-7ed9-4fb7-af62-52e9373195bf', 'volume_name': 'volume-d2a1c319-7ed9-4fb7-af62-52e9373195bf', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:00.203971 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (794d13e6-c858-45cb-83fd-2d9476567d8f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:00.210050 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (794d13e6-c858-45cb-83fd-2d9476567d8f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:00.216017 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6ed2752b-cba7-4573-a419-fa4acab5b2d2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:00.217438 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Volume d2a1c319-7ed9-4fb7-af62-52e9373195bf: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d2a1c319-7ed9-4fb7-af62-52e9373195bf', 'volume_size': 1} Sep 17 20:55:00.217832 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] creating volume 'volume-d2a1c319-7ed9-4fb7-af62-52e9373195bf' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 20:55:00.218659 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:55:00.460924 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 20:55:00.461707 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 20:55:00.465853 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6ed2752b-cba7-4573-a419-fa4acab5b2d2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d2a1c319-7ed9-4fb7-af62-52e9373195bf', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:00.475582 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c128cbcc-c770-48c1-b87d-4a6350417c14) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:00.498514 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Volume volume-d2a1c319-7ed9-4fb7-af62-52e9373195bf (d2a1c319-7ed9-4fb7-af62-52e9373195bf): created successfully Sep 17 20:55:00.505914 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c128cbcc-c770-48c1-b87d-4a6350417c14) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:00.526194 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Flow 'volume_create_manager' (a1b1d973-cfac-4294-ba38-f9241c7c4bcb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:55:00.554985 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-20e552a8-e705-4fcb-9e88-b68b580e7a49 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Created volume successfully. Sep 17 20:55:01.039015 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:55:01.039771 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:55:01.123628 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:55:01.202307 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:55:02.038618 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=10,cinder:UPDATE=4,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:02.058831 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 20:55:02.106453 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Flow 'volume_create_manager' (6b5b27fc-4c45-4f4b-8be5-f058dcc4d31a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:55:02.131240 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3e2335e3-8054-4d8f-932a-7514dd76f002) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:02.186522 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3e2335e3-8054-4d8f-932a-7514dd76f002) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T20:55:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-1885418955',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=99848cf4-d1b1-4750-bd2d-84def7a9a754,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='822801d3963d45c1b4176259fb5c5b19',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T20:55:02Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T20:55:02Z,use_quota=True,user_id='55e8e491f8844f7696f93d8ec5a7171b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:02.194660 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fbdd5c59-acce-49d8-b709-a58a44f3c2bc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:02.208210 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fbdd5c59-acce-49d8-b709-a58a44f3c2bc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:02.220973 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4ad0bea2-4b72-41f8-8152-d4704dd86d6a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:02.430594 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4ad0bea2-4b72-41f8-8152-d4704dd86d6a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '99848cf4-d1b1-4750-bd2d-84def7a9a754', 'volume_name': 'volume-99848cf4-d1b1-4750-bd2d-84def7a9a754', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:02.449856 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a2b84254-7630-424c-83f6-36864823279b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:02.476594 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a2b84254-7630-424c-83f6-36864823279b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:02.493545 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (17b6ce7a-f08e-4267-8347-550831b3b574) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:02.499327 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Volume 99848cf4-d1b1-4750-bd2d-84def7a9a754: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-99848cf4-d1b1-4750-bd2d-84def7a9a754', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 20:55:02.501499 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Cloning 99848cf4-d1b1-4750-bd2d-84def7a9a754 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 20:55:02.502329 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 20:55:02.529794 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.025s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 20:55:02.566097 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 20, 55, 2, 548292)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 20:55:02.566608 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 20:55:02.567023 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 20:55:02.582486 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.055s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 20:55:02.584682 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 20:55:02.599628 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 20, 55, 2, 592453)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 20:55:02.600095 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 20:55:02.600660 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 20:55:02.665559 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:55:02.802154 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] creating snapshot='volume-99848cf4-d1b1-4750-bd2d-84def7a9a754.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 20:55:03.283791 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-99848cf4-d1b1-4750-bd2d-84def7a9a754.clone_snap' to 'volume-99848cf4-d1b1-4750-bd2d-84def7a9a754' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 20:55:03.712101 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 20:55:03.713924 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 20:55:03.714656 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 20:55:03.772129 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 20:55:03.772927 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Marking volume 99848cf4-d1b1-4750-bd2d-84def7a9a754 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 20:55:03.787749 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to 99848cf4-d1b1-4750-bd2d-84def7a9a754. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 20:55:03.788579 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Creating volume glance metadata for volume 99848cf4-d1b1-4750-bd2d-84def7a9a754 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 20:55:04.008879 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (17b6ce7a-f08e-4267-8347-550831b3b574) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-99848cf4-d1b1-4750-bd2d-84def7a9a754', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:04.016944 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (12545821-9f43-43f5-abfa-01c4b2f0951c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:04.104815 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-019c656a-b681-4c17-85b5-7d75aeb9e0d9 req-aa3c1352-ea30-4cdb-8a00-8bf669c5d84f tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:55:04.148764 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Volume volume-99848cf4-d1b1-4750-bd2d-84def7a9a754 (99848cf4-d1b1-4750-bd2d-84def7a9a754): created successfully Sep 17 20:55:04.161190 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (12545821-9f43-43f5-abfa-01c4b2f0951c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:04.177390 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Flow 'volume_create_manager' (6b5b27fc-4c45-4f4b-8be5-f058dcc4d31a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:55:04.256640 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-9e506a6a-6ed9-4271-b1d8-1456721fa4d4 tempest-TaggedBootDevicesTest-356253908 None] Created volume successfully. Sep 17 20:55:05.701291 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 20:55:05.745386 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Flow 'volume_create_manager' (0e178c16-81e1-400f-9129-a480ab485157) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:55:05.759944 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c9d6fcb4-0478-4036-a4a1-2305f4c73e30) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:05.784703 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c9d6fcb4-0478-4036-a4a1-2305f4c73e30) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T20:55:05Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-1994594219',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=39a25695-6a20-4519-b4cc-233d172fde0a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='822801d3963d45c1b4176259fb5c5b19',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T20:55:06Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T20:55:06Z,use_quota=True,user_id='55e8e491f8844f7696f93d8ec5a7171b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:05.799049 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8acd8acb-351e-475f-83a7-0663a6e760e0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:05.803321 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8acd8acb-351e-475f-83a7-0663a6e760e0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:05.818134 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0a147a4b-ca43-4c54-abd8-0216f8a95229) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:05.822720 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0a147a4b-ca43-4c54-abd8-0216f8a95229) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '39a25695-6a20-4519-b4cc-233d172fde0a', 'volume_name': 'volume-39a25695-6a20-4519-b4cc-233d172fde0a', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:05.838840 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (10c0663a-8983-4f54-8833-85f56e4e3a82) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:05.843222 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (10c0663a-8983-4f54-8833-85f56e4e3a82) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:05.860677 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4688036e-04d9-49e9-9b22-a069626f84ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:05.861944 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Volume 39a25695-6a20-4519-b4cc-233d172fde0a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-39a25695-6a20-4519-b4cc-233d172fde0a', 'volume_size': 1} Sep 17 20:55:05.862368 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] creating volume 'volume-39a25695-6a20-4519-b4cc-233d172fde0a' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 20:55:05.863166 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:55:06.061742 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-019c656a-b681-4c17-85b5-7d75aeb9e0d9 req-aa3c1352-ea30-4cdb-8a00-8bf669c5d84f tempest-LiveAutoBlockMigrationV225Test-1505099207 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.957s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:55:06.062681 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-019c656a-b681-4c17-85b5-7d75aeb9e0d9 req-aa3c1352-ea30-4cdb-8a00-8bf669c5d84f tempest-LiveAutoBlockMigrationV225Test-1505099207 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-d2a1c319-7ed9-4fb7-af62-52e9373195bf', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'd2a1c319-7ed9-4fb7-af62-52e9373195bf', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:55:06.092023 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 20:55:06.092720 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 20:55:06.103551 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4688036e-04d9-49e9-9b22-a069626f84ba) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-39a25695-6a20-4519-b4cc-233d172fde0a', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:06.112374 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0ac0cba5-cd5f-4133-b51a-4b1acbf12216) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:06.139100 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Volume volume-39a25695-6a20-4519-b4cc-233d172fde0a (39a25695-6a20-4519-b4cc-233d172fde0a): created successfully Sep 17 20:55:06.143041 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0ac0cba5-cd5f-4133-b51a-4b1acbf12216) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:06.152559 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Flow 'volume_create_manager' (0e178c16-81e1-400f-9129-a480ab485157) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:55:06.165553 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a4140705-1053-4395-aea6-10f2df1f0d6e tempest-TaggedBootDevicesTest-356253908 None] Created volume successfully. Sep 17 20:55:06.172879 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-019c656a-b681-4c17-85b5-7d75aeb9e0d9 req-aa3c1352-ea30-4cdb-8a00-8bf669c5d84f tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Connection info returned from driver {'name': 'volumes/volume-d2a1c319-7ed9-4fb7-af62-52e9373195bf', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'd2a1c319-7ed9-4fb7-af62-52e9373195bf', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '1087adf9-bca6-4720-ab53-06865ab5afcd'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:55:06.274263 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-019c656a-b681-4c17-85b5-7d75aeb9e0d9 req-aa3c1352-ea30-4cdb-8a00-8bf669c5d84f tempest-LiveAutoBlockMigrationV225Test-1505099207 None] attachment_update completed successfully. Sep 17 20:55:07.432093 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 20:55:07.464080 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Flow 'volume_create_manager' (9ce3f83c-838a-4600-baa9-7bfa874716d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:55:07.470483 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (19000d6b-101e-4a6e-a65f-f666187df993) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:07.490135 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (19000d6b-101e-4a6e-a65f-f666187df993) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T20:55:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-2012965254',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1a721e0a-0c66-4cab-9a36-ed78def77377,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='822801d3963d45c1b4176259fb5c5b19',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T20:55:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T20:55:07Z,use_quota=True,user_id='55e8e491f8844f7696f93d8ec5a7171b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:07.497441 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1ad81c34-47c7-4c44-a7a0-e393156b6342) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:07.503135 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1ad81c34-47c7-4c44-a7a0-e393156b6342) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:07.510544 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (661fd413-80e4-4b02-bf8e-a7f847b738a1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:07.519933 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (661fd413-80e4-4b02-bf8e-a7f847b738a1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1a721e0a-0c66-4cab-9a36-ed78def77377', 'volume_name': 'volume-1a721e0a-0c66-4cab-9a36-ed78def77377', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:07.527250 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0262bb70-4865-47a2-a48b-72c21b6517be) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:07.532078 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0262bb70-4865-47a2-a48b-72c21b6517be) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:07.538981 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (52259c65-32d2-4e95-8da9-ce1cc83c80fa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:07.540306 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Volume 1a721e0a-0c66-4cab-9a36-ed78def77377: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1a721e0a-0c66-4cab-9a36-ed78def77377', 'volume_size': 1} Sep 17 20:55:07.540858 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] creating volume 'volume-1a721e0a-0c66-4cab-9a36-ed78def77377' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 20:55:07.541701 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:55:07.685347 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 20:55:07.685770 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 20:55:07.689536 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (52259c65-32d2-4e95-8da9-ce1cc83c80fa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1a721e0a-0c66-4cab-9a36-ed78def77377', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:07.694298 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5786d5a4-4a95-4ce8-9e9b-9dbee2cc3e68) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:07.723215 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Volume volume-1a721e0a-0c66-4cab-9a36-ed78def77377 (1a721e0a-0c66-4cab-9a36-ed78def77377): created successfully Sep 17 20:55:07.726971 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5786d5a4-4a95-4ce8-9e9b-9dbee2cc3e68) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:07.732345 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Flow 'volume_create_manager' (9ce3f83c-838a-4600-baa9-7bfa874716d1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:55:07.757881 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-89a5244d-a8a0-4985-80b5-51dc4e320160 tempest-TaggedBootDevicesTest-356253908 None] Created volume successfully. Sep 17 20:55:16.501364 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=49,cinder:UPDATE=9,cinder:INSERT=3,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:16.523947 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-79ebc3af-8b5f-4a0b-a77d-3fe526c9ee78 req-5cd883fd-63e1-4b5b-ba01-585b52cabf01 tempest-LiveAutoBlockMigrationV225Test-832919131 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:55:17.955959 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-79ebc3af-8b5f-4a0b-a77d-3fe526c9ee78 req-5cd883fd-63e1-4b5b-ba01-585b52cabf01 tempest-LiveAutoBlockMigrationV225Test-832919131 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.432s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:55:17.957189 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-79ebc3af-8b5f-4a0b-a77d-3fe526c9ee78 req-5cd883fd-63e1-4b5b-ba01-585b52cabf01 tempest-LiveAutoBlockMigrationV225Test-832919131 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-d2a1c319-7ed9-4fb7-af62-52e9373195bf', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'd2a1c319-7ed9-4fb7-af62-52e9373195bf', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:55:17.990123 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-79ebc3af-8b5f-4a0b-a77d-3fe526c9ee78 req-5cd883fd-63e1-4b5b-ba01-585b52cabf01 tempest-LiveAutoBlockMigrationV225Test-832919131 None] Connection info returned from driver {'name': 'volumes/volume-d2a1c319-7ed9-4fb7-af62-52e9373195bf', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'd2a1c319-7ed9-4fb7-af62-52e9373195bf', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '408489e1-5c29-464c-87bc-b27da774d070'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:55:18.110558 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-79ebc3af-8b5f-4a0b-a77d-3fe526c9ee78 req-5cd883fd-63e1-4b5b-ba01-585b52cabf01 tempest-LiveAutoBlockMigrationV225Test-832919131 None] attachment_update completed successfully. Sep 17 20:55:33.821419 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=51,cinder:INSERT=9,cinder:UPDATE=12 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:33.825713 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-79ebc3af-8b5f-4a0b-a77d-3fe526c9ee78 req-b38e3053-eb80-4dbc-be97-bb617ed0dc1b tempest-LiveAutoBlockMigrationV225Test-832919131 None] Terminate volume connection completed successfully. Sep 17 20:55:33.826138 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-79ebc3af-8b5f-4a0b-a77d-3fe526c9ee78 req-b38e3053-eb80-4dbc-be97-bb617ed0dc1b tempest-LiveAutoBlockMigrationV225Test-832919131 None] Deleting attachment 1087adf9-bca6-4720-ab53-06865ab5afcd. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 20:55:35.483018 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-0ca38813-ecc4-4679-b1ea-589e79e883a8 tempest-TaggedBootDevicesTest-356253908 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:55:36.611095 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-0ca38813-ecc4-4679-b1ea-589e79e883a8 tempest-TaggedBootDevicesTest-356253908 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.128s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:55:36.615137 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-0ca38813-ecc4-4679-b1ea-589e79e883a8 tempest-TaggedBootDevicesTest-356253908 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-99848cf4-d1b1-4750-bd2d-84def7a9a754', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '99848cf4-d1b1-4750-bd2d-84def7a9a754', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:55:36.654238 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-0ca38813-ecc4-4679-b1ea-589e79e883a8 tempest-TaggedBootDevicesTest-356253908 None] Connection info returned from driver {'name': 'volumes/volume-99848cf4-d1b1-4750-bd2d-84def7a9a754', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '99848cf4-d1b1-4750-bd2d-84def7a9a754', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '7eb215eb-8b80-4fc5-a916-86ba7a7c14fb'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:55:36.788579 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-0ca38813-ecc4-4679-b1ea-589e79e883a8 tempest-TaggedBootDevicesTest-356253908 None] attachment_update completed successfully. Sep 17 20:55:37.205574 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-33fecffc-ad59-4cd1-b439-59207528c60d tempest-ServersTestBootFromVolume-594505527 None] Terminate volume connection completed successfully. Sep 17 20:55:37.209762 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-33fecffc-ad59-4cd1-b439-59207528c60d tempest-ServersTestBootFromVolume-594505527 None] Deleting attachment 57aef2c2-1c7b-4eda-b1fd-b8b2328bc398. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 20:55:37.632145 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-e1604d72-13a9-4a3a-87e1-fe2161a0b16e tempest-TaggedBootDevicesTest-356253908 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:55:37.760273 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] Acquiring lock "cinder-d32359bf-ad08-4517-9759-3f42907f23c0-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 20:55:37.786700 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] Lock "cinder-d32359bf-ad08-4517-9759-3f42907f23c0-delete_volume" acquired by "delete_volume" :: waited 0.027s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 20:55:37.883684 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:55:37.980716 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 20:55:37.981344 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] volume has clone snapshot(s) {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1378}} Sep 17 20:55:37.984588 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 20:55:38.325289 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] Created reservations ['787ad5f0-ebd7-48d7-953f-4d8bdf1a9be1', '73e1161a-fa48-49e4-8091-3c4f07149ea5', '7fa4fb5b-e00d-45ba-bba3-8572eb52511d', '1900e84c-191f-4b33-b410-d20a1f0fe7d9'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 20:55:38.460457 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:55:38.583279 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:55:38.702905 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:55:38.708670 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] Deleted volume successfully. Sep 17 20:55:38.733603 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-f46b18a5-31e1-41f7-af8a-85042e0bc7bf req-4e89fdef-fcae-4783-a782-8e8c2fd2ab5e tempest-ServersTestBootFromVolume-594505527 None] Lock "cinder-d32359bf-ad08-4517-9759-3f42907f23c0-delete_volume" released by "delete_volume" :: held 0.947s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 20:55:40.119440 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-e1604d72-13a9-4a3a-87e1-fe2161a0b16e tempest-TaggedBootDevicesTest-356253908 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 2.487s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:55:40.120135 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-e1604d72-13a9-4a3a-87e1-fe2161a0b16e tempest-TaggedBootDevicesTest-356253908 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-39a25695-6a20-4519-b4cc-233d172fde0a', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '39a25695-6a20-4519-b4cc-233d172fde0a', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:55:40.162808 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-e1604d72-13a9-4a3a-87e1-fe2161a0b16e tempest-TaggedBootDevicesTest-356253908 None] Connection info returned from driver {'name': 'volumes/volume-39a25695-6a20-4519-b4cc-233d172fde0a', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '39a25695-6a20-4519-b4cc-233d172fde0a', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'a88e6abe-3519-4581-b8d1-391709977707'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:55:40.268344 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-e1604d72-13a9-4a3a-87e1-fe2161a0b16e tempest-TaggedBootDevicesTest-356253908 None] attachment_update completed successfully. Sep 17 20:55:40.805327 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-fe1befef-450e-4d3d-8678-9ff0227c02f7 tempest-TaggedBootDevicesTest-356253908 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:55:42.217434 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-fe1befef-450e-4d3d-8678-9ff0227c02f7 tempest-TaggedBootDevicesTest-356253908 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.412s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:55:42.218931 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-fe1befef-450e-4d3d-8678-9ff0227c02f7 tempest-TaggedBootDevicesTest-356253908 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1a721e0a-0c66-4cab-9a36-ed78def77377', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '1a721e0a-0c66-4cab-9a36-ed78def77377', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:55:42.269379 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-fe1befef-450e-4d3d-8678-9ff0227c02f7 tempest-TaggedBootDevicesTest-356253908 None] Connection info returned from driver {'name': 'volumes/volume-1a721e0a-0c66-4cab-9a36-ed78def77377', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '1a721e0a-0c66-4cab-9a36-ed78def77377', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '985d11cc-c1ab-48b2-9279-44882378589a'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:55:42.355372 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-dfb835c2-314b-41c9-b67c-faba76296ae9 req-fe1befef-450e-4d3d-8678-9ff0227c02f7 tempest-TaggedBootDevicesTest-356253908 None] attachment_update completed successfully. Sep 17 20:55:49.322418 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=54,cinder:UPDATE=15,cinder:INSERT=3,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:55:49.341939 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-43c5365a-a21d-47af-9aa0-d45fa44aa3f7 req-d67b2a17-d47d-4d04-b26a-b12cb235975d tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Terminate volume connection completed successfully. Sep 17 20:55:49.342375 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-43c5365a-a21d-47af-9aa0-d45fa44aa3f7 req-d67b2a17-d47d-4d04-b26a-b12cb235975d tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Deleting attachment 408489e1-5c29-464c-87bc-b27da774d070. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 20:55:51.012928 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 20:55:51.039879 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Flow 'volume_create_manager' (26e5f086-a426-4d54-81d1-d12b52bee779) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:55:51.048011 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (161e7e3b-fe71-4e4a-a5a9-5d5304eda7d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:51.078100 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (161e7e3b-fe71-4e4a-a5a9-5d5304eda7d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T20:55:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1814612366',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=73bec761-6a0d-444f-9c7c-1504a4ca70c6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='7dc8bc81c2d74bae9612f7c9807350a7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T20:55:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T20:55:51Z,use_quota=True,user_id='f1134231b4a545fdb136a5204cb4f289',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:51.084900 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8f364123-e098-4cd0-a233-bfb0eb3c9b14) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:51.090140 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8f364123-e098-4cd0-a233-bfb0eb3c9b14) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:51.095913 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2c23e661-7fdd-40d1-9609-31f323660ea2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:51.201854 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2c23e661-7fdd-40d1-9609-31f323660ea2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'volume_name': 'volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:51.206656 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d826003a-68c5-4dcb-a122-b4aa1a49643a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:51.210896 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d826003a-68c5-4dcb-a122-b4aa1a49643a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:51.215681 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (32696598-3cf6-41cb-99fe-bd4311c7388d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:51.216997 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Volume 73bec761-6a0d-444f-9c7c-1504a4ca70c6: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 20:55:51.217877 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Cloning 73bec761-6a0d-444f-9c7c-1504a4ca70c6 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 20:55:51.218860 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 20:55:51.258087 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.039s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 20:55:51.279734 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 20, 55, 51, 263257)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 20:55:51.280189 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 20:55:51.282906 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 20:55:51.290419 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.032s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 20:55:51.292643 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 20:55:51.314429 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 20, 55, 51, 306094)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 20:55:51.319511 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 20:55:51.320732 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 20:55:51.365394 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:55:51.504085 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] creating snapshot='volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 20:55:52.113206 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6.clone_snap' to 'volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 20:55:52.340002 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 20:55:52.342573 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 20:55:52.342968 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 20:55:52.419956 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 20:55:52.421357 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Marking volume 73bec761-6a0d-444f-9c7c-1504a4ca70c6 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 20:55:52.434360 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to 73bec761-6a0d-444f-9c7c-1504a4ca70c6. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 20:55:52.435282 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Creating volume glance metadata for volume 73bec761-6a0d-444f-9c7c-1504a4ca70c6 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 20:55:52.516819 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (32696598-3cf6-41cb-99fe-bd4311c7388d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:52.523924 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (70e15e71-8570-4c08-af96-1f105331e3db) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:55:52.539795 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Volume volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6 (73bec761-6a0d-444f-9c7c-1504a4ca70c6): created successfully Sep 17 20:55:52.545431 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (70e15e71-8570-4c08-af96-1f105331e3db) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:55:52.552063 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Flow 'volume_create_manager' (26e5f086-a426-4d54-81d1-d12b52bee779) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:55:52.587301 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f21b471d-a6d2-401b-96c3-87fc000782b0 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Created volume successfully. Sep 17 20:55:57.340019 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-8502c882-40b6-40cb-b768-322ff38ebe20 req-74e0c4db-7136-4e4b-8151-6b88d72e3759 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:55:58.490347 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-8502c882-40b6-40cb-b768-322ff38ebe20 req-74e0c4db-7136-4e4b-8151-6b88d72e3759 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.151s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:55:58.491467 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-8502c882-40b6-40cb-b768-322ff38ebe20 req-74e0c4db-7136-4e4b-8151-6b88d72e3759 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:55:58.547544 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-8502c882-40b6-40cb-b768-322ff38ebe20 req-74e0c4db-7136-4e4b-8151-6b88d72e3759 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Connection info returned from driver {'name': 'volumes/volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '3b14ea46-d00d-4e2c-8922-2e954da61acc'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:55:58.635689 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-8502c882-40b6-40cb-b768-322ff38ebe20 req-74e0c4db-7136-4e4b-8151-6b88d72e3759 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] attachment_update completed successfully. Sep 17 20:56:01.040152 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:56:01.041003 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:56:01.142789 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:56:01.205354 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:56:01.250134 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 20:56:01.284161 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Flow 'volume_create_manager' (5b3a66c7-ad79-4e70-8d60-df63607d76dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:56:01.291536 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (490510b6-ed5c-4359-9a82-5ec58dcf870f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:01.313597 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (490510b6-ed5c-4359-9a82-5ec58dcf870f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T20:56:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveMigrationTest-volume-796052317',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=39a2a629-4cdb-497c-8f9e-729c11354b4b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f6b44bc44e7c4a998bfd99e7d11e99fc',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T20:56:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T20:56:01Z,use_quota=True,user_id='9c2b746b7c194596808a7b0c736f3b53',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:01.320006 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (28b61ee2-58dc-4fd8-9689-0ab8627f37da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:01.328208 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (28b61ee2-58dc-4fd8-9689-0ab8627f37da) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:01.333604 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3f0a8d8d-7e39-4370-a4a3-31cf04ff12a5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:01.339028 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3f0a8d8d-7e39-4370-a4a3-31cf04ff12a5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '39a2a629-4cdb-497c-8f9e-729c11354b4b', 'volume_name': 'volume-39a2a629-4cdb-497c-8f9e-729c11354b4b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:01.344232 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e74dbbc5-82c9-49a0-863e-eeb93c8d10f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:01.349773 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e74dbbc5-82c9-49a0-863e-eeb93c8d10f9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:01.354738 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7ca38877-400f-4bdc-8a0b-6931fb18b4d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:01.356199 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Volume 39a2a629-4cdb-497c-8f9e-729c11354b4b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-39a2a629-4cdb-497c-8f9e-729c11354b4b', 'volume_size': 1} Sep 17 20:56:01.356790 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] creating volume 'volume-39a2a629-4cdb-497c-8f9e-729c11354b4b' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 20:56:01.357913 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:56:01.487444 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 20:56:01.487902 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 20:56:01.499765 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7ca38877-400f-4bdc-8a0b-6931fb18b4d7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-39a2a629-4cdb-497c-8f9e-729c11354b4b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:01.505227 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e18be57b-21ee-42dc-9332-5bc33dac74a5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:01.522445 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Volume volume-39a2a629-4cdb-497c-8f9e-729c11354b4b (39a2a629-4cdb-497c-8f9e-729c11354b4b): created successfully Sep 17 20:56:01.527374 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e18be57b-21ee-42dc-9332-5bc33dac74a5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:01.533924 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Flow 'volume_create_manager' (5b3a66c7-ad79-4e70-8d60-df63607d76dd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:56:01.538546 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=80,cinder:UPDATE=19,cinder:INSERT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:01.552100 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-586f2ee8-b355-4859-be1f-e38174fd5158 tempest-LiveMigrationTest-128259988 None] Created volume successfully. Sep 17 20:56:03.026633 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-1161023c-d8bd-4255-823e-e9270b1eec97 req-cb7b7643-2ec6-4267-bf71-a865b1a570cf tempest-LiveMigrationTest-128259988 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:56:04.357387 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-1161023c-d8bd-4255-823e-e9270b1eec97 req-cb7b7643-2ec6-4267-bf71-a865b1a570cf tempest-LiveMigrationTest-128259988 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.331s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:56:04.358220 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-1161023c-d8bd-4255-823e-e9270b1eec97 req-cb7b7643-2ec6-4267-bf71-a865b1a570cf tempest-LiveMigrationTest-128259988 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-39a2a629-4cdb-497c-8f9e-729c11354b4b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '39a2a629-4cdb-497c-8f9e-729c11354b4b', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:56:04.413126 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1161023c-d8bd-4255-823e-e9270b1eec97 req-cb7b7643-2ec6-4267-bf71-a865b1a570cf tempest-LiveMigrationTest-128259988 None] Connection info returned from driver {'name': 'volumes/volume-39a2a629-4cdb-497c-8f9e-729c11354b4b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '39a2a629-4cdb-497c-8f9e-729c11354b4b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '6a3f14a4-3324-4850-866a-0aa3a9c1bb69'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:56:04.517957 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-1161023c-d8bd-4255-823e-e9270b1eec97 req-cb7b7643-2ec6-4267-bf71-a865b1a570cf tempest-LiveMigrationTest-128259988 None] attachment_update completed successfully. Sep 17 20:56:13.620163 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-3bb407aa-2d2c-424f-bbdb-d76a64f69a25 req-cd4cc50e-2ec7-40c4-bd65-a1775eb215f9 tempest-LiveMigrationTest-703140491 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:56:16.039838 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-3bb407aa-2d2c-424f-bbdb-d76a64f69a25 req-cd4cc50e-2ec7-40c4-bd65-a1775eb215f9 tempest-LiveMigrationTest-703140491 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 2.419s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:56:16.040801 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-3bb407aa-2d2c-424f-bbdb-d76a64f69a25 req-cd4cc50e-2ec7-40c4-bd65-a1775eb215f9 tempest-LiveMigrationTest-703140491 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-39a2a629-4cdb-497c-8f9e-729c11354b4b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '39a2a629-4cdb-497c-8f9e-729c11354b4b', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:56:16.129280 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-3bb407aa-2d2c-424f-bbdb-d76a64f69a25 req-cd4cc50e-2ec7-40c4-bd65-a1775eb215f9 tempest-LiveMigrationTest-703140491 None] Connection info returned from driver {'name': 'volumes/volume-39a2a629-4cdb-497c-8f9e-729c11354b4b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '39a2a629-4cdb-497c-8f9e-729c11354b4b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'bc70a03a-3d08-456c-bf7b-357d14ff0315'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:56:16.354237 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-3bb407aa-2d2c-424f-bbdb-d76a64f69a25 req-cd4cc50e-2ec7-40c4-bd65-a1775eb215f9 tempest-LiveMigrationTest-703140491 None] attachment_update completed successfully. Sep 17 20:56:21.411539 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=59,cinder:UPDATE=11,cinder:INSERT=12,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:30.856196 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-bbffe54a-072a-47ce-a812-c412c282b7a9 req-aaa89d74-7410-4a5a-94da-84abfa19550e tempest-LiveAutoBlockMigrationV225Test-832919131 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:56:32.596955 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-bbffe54a-072a-47ce-a812-c412c282b7a9 req-aaa89d74-7410-4a5a-94da-84abfa19550e tempest-LiveAutoBlockMigrationV225Test-832919131 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.741s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:56:32.597826 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-bbffe54a-072a-47ce-a812-c412c282b7a9 req-aaa89d74-7410-4a5a-94da-84abfa19550e tempest-LiveAutoBlockMigrationV225Test-832919131 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:56:32.601182 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=33,cinder:UPDATE=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:32.631837 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-bbffe54a-072a-47ce-a812-c412c282b7a9 req-aaa89d74-7410-4a5a-94da-84abfa19550e tempest-LiveAutoBlockMigrationV225Test-832919131 None] Connection info returned from driver {'name': 'volumes/volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '73bec761-6a0d-444f-9c7c-1504a4ca70c6', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'aaafda3c-8c7a-4d6e-8d9e-db9adb6b5b7a'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:56:32.722391 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-bbffe54a-072a-47ce-a812-c412c282b7a9 req-aaa89d74-7410-4a5a-94da-84abfa19550e tempest-LiveAutoBlockMigrationV225Test-832919131 None] attachment_update completed successfully. Sep 17 20:56:33.929008 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-3bb407aa-2d2c-424f-bbdb-d76a64f69a25 req-d08f8ee5-c802-45e6-8889-df08cecb2755 tempest-LiveMigrationTest-703140491 None] Terminate volume connection completed successfully. Sep 17 20:56:33.929768 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-3bb407aa-2d2c-424f-bbdb-d76a64f69a25 req-d08f8ee5-c802-45e6-8889-df08cecb2755 tempest-LiveMigrationTest-703140491 None] Deleting attachment 6a3f14a4-3324-4850-866a-0aa3a9c1bb69. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 20:56:43.144002 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-67678a09-9d6f-4a99-8ab1-2f1e453131cd req-d3bb645d-fa41-4803-b109-00e3251622f9 tempest-LiveMigrationTest-128259988 None] Terminate volume connection completed successfully. Sep 17 20:56:43.146669 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-67678a09-9d6f-4a99-8ab1-2f1e453131cd req-d3bb645d-fa41-4803-b109-00e3251622f9 tempest-LiveMigrationTest-128259988 None] Deleting attachment bc70a03a-3d08-456c-bf7b-357d14ff0315. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 20:56:44.483933 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 20:56:44.508839 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Flow 'volume_create_manager' (dea9f538-f78b-4f69-b4a0-f98e5c52a031) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:56:44.517330 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8874bc80-c11b-4a0d-b73e-1eb338403091) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:44.540635 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8874bc80-c11b-4a0d-b73e-1eb338403091) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T20:56:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1823139922',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=67568702-e073-4ebf-aeae-310160ebcb1e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f6b44bc44e7c4a998bfd99e7d11e99fc',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T20:56:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T20:56:44Z,use_quota=True,user_id='9c2b746b7c194596808a7b0c736f3b53',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:44.547699 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b840e210-b354-4b85-9f8d-20a19d91e00b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:44.553984 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b840e210-b354-4b85-9f8d-20a19d91e00b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:44.562430 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (66a73c83-bdd0-44ac-80b0-1bbd24cc5cf2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:44.721405 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (66a73c83-bdd0-44ac-80b0-1bbd24cc5cf2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '67568702-e073-4ebf-aeae-310160ebcb1e', 'volume_name': 'volume-67568702-e073-4ebf-aeae-310160ebcb1e', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:44.734854 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9191597d-437e-4f43-8a5c-8eb1c1231540) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:44.747893 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9191597d-437e-4f43-8a5c-8eb1c1231540) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:44.762356 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7d61b284-419a-4eec-ba18-a95a4c12ee4c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:44.763968 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Volume 67568702-e073-4ebf-aeae-310160ebcb1e: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-67568702-e073-4ebf-aeae-310160ebcb1e', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 20:56:44.764743 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Cloning 67568702-e073-4ebf-aeae-310160ebcb1e from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 20:56:44.765937 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 20:56:44.785006 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.019s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 20:56:44.801685 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 20, 56, 44, 791777)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 20:56:44.802259 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 20:56:44.802764 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 20:56:44.808740 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.024s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 20:56:44.809589 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 20:56:44.818058 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 20, 56, 44, 814764)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 20:56:44.818758 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 20:56:44.819348 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 20:56:44.833934 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:56:44.963309 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] creating snapshot='volume-67568702-e073-4ebf-aeae-310160ebcb1e.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 20:56:45.380281 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-67568702-e073-4ebf-aeae-310160ebcb1e.clone_snap' to 'volume-67568702-e073-4ebf-aeae-310160ebcb1e' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 20:56:45.600479 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 20:56:45.601753 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 20:56:45.602469 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 20:56:45.666066 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 20:56:45.670093 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Marking volume 67568702-e073-4ebf-aeae-310160ebcb1e as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 20:56:45.699743 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to 67568702-e073-4ebf-aeae-310160ebcb1e. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 20:56:45.700246 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Creating volume glance metadata for volume 67568702-e073-4ebf-aeae-310160ebcb1e backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 20:56:45.772207 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-bbffe54a-072a-47ce-a812-c412c282b7a9 req-1627ede7-49a5-4b72-abaf-ebd76e86c249 tempest-LiveAutoBlockMigrationV225Test-832919131 None] Terminate volume connection completed successfully. Sep 17 20:56:45.773030 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-bbffe54a-072a-47ce-a812-c412c282b7a9 req-1627ede7-49a5-4b72-abaf-ebd76e86c249 tempest-LiveAutoBlockMigrationV225Test-832919131 None] Deleting attachment 3b14ea46-d00d-4e2c-8922-2e954da61acc. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 20:56:45.867327 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7d61b284-419a-4eec-ba18-a95a4c12ee4c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-67568702-e073-4ebf-aeae-310160ebcb1e', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:45.872204 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2887e1a0-314c-4c9d-9589-4f72b700662a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:56:45.899693 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Volume volume-67568702-e073-4ebf-aeae-310160ebcb1e (67568702-e073-4ebf-aeae-310160ebcb1e): created successfully Sep 17 20:56:45.911980 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2887e1a0-314c-4c9d-9589-4f72b700662a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:56:45.932572 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Flow 'volume_create_manager' (dea9f538-f78b-4f69-b4a0-f98e5c52a031) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:56:45.955098 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6979796e-a7d5-4ecf-ab96-1bb9eb782353 tempest-LiveMigrationTest-128259988 None] Created volume successfully. Sep 17 20:56:53.730340 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=34,cinder:UPDATE=6,cinder:INSERT=5,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:56:53.742429 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-35b74a39-b982-47a5-be8e-b8142640150b req-503e8ef4-21b1-4c70-b04c-775891ca3eef tempest-LiveMigrationTest-128259988 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:56:54.850322 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-35b74a39-b982-47a5-be8e-b8142640150b req-503e8ef4-21b1-4c70-b04c-775891ca3eef tempest-LiveMigrationTest-128259988 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.105s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:56:54.852347 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-35b74a39-b982-47a5-be8e-b8142640150b req-503e8ef4-21b1-4c70-b04c-775891ca3eef tempest-LiveMigrationTest-128259988 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-67568702-e073-4ebf-aeae-310160ebcb1e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '67568702-e073-4ebf-aeae-310160ebcb1e', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:56:54.884310 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-35b74a39-b982-47a5-be8e-b8142640150b req-503e8ef4-21b1-4c70-b04c-775891ca3eef tempest-LiveMigrationTest-128259988 None] Connection info returned from driver {'name': 'volumes/volume-67568702-e073-4ebf-aeae-310160ebcb1e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '67568702-e073-4ebf-aeae-310160ebcb1e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '0ac9f7b4-1309-4212-b108-62093f4c19dc'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:56:54.960107 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-35b74a39-b982-47a5-be8e-b8142640150b req-503e8ef4-21b1-4c70-b04c-775891ca3eef tempest-LiveMigrationTest-128259988 None] attachment_update completed successfully. Sep 17 20:57:01.037545 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:57:01.038443 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:01.075458 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:01.114258 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:57:03.020701 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=8,cinder:SELECT=49,cinder:INSERT=7 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:03.033788 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-d8b647bd-4eb7-4901-b8ed-5171f2bf3151 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Terminate volume connection completed successfully. Sep 17 20:57:03.034510 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-d8b647bd-4eb7-4901-b8ed-5171f2bf3151 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Deleting attachment aaafda3c-8c7a-4d6e-8d9e-db9adb6b5b7a. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 20:57:03.946724 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Acquiring lock "cinder-73bec761-6a0d-444f-9c7c-1504a4ca70c6-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 20:57:03.962174 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Lock "cinder-73bec761-6a0d-444f-9c7c-1504a4ca70c6-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 20:57:04.015477 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:04.123859 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 20:57:04.134797 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] deleting rbd volume volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 20:57:04.363192 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 20:57:04.396283 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] deleting parent snapshot volume-73bec761-6a0d-444f-9c7c-1504a4ca70c6.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 20:57:05.410239 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Acquiring lock "cinder-d2a1c319-7ed9-4fb7-af62-52e9373195bf-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 20:57:05.433596 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Lock "cinder-d2a1c319-7ed9-4fb7-af62-52e9373195bf-delete_volume" acquired by "delete_volume" :: waited 0.024s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 20:57:05.480185 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:05.541780 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 20:57:05.542261 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Volume volume-d2a1c319-7ed9-4fb7-af62-52e9373195bf is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 20:57:05.547161 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] deleting rbd volume volume-d2a1c319-7ed9-4fb7-af62-52e9373195bf {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 20:57:05.549537 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Created reservations ['93db4583-1a76-447f-a242-5ea48e68a472', '58838d96-88a8-498f-998b-1eca9c6c374e', 'bacb4a22-3082-49cb-8992-e1fd8f7baf20', '407c4728-a5bf-499d-9733-9b190c245644'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 20:57:05.615879 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:05.662614 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:05.734220 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:57:05.748576 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Deleted volume successfully. Sep 17 20:57:05.776698 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-ff96786e-cf6b-40e0-99d7-b7169cf7a8b8 req-0e834ffb-54f2-47af-b18a-abeb17d4ada4 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Lock "cinder-73bec761-6a0d-444f-9c7c-1504a4ca70c6-delete_volume" released by "delete_volume" :: held 1.814s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 20:57:05.827618 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Created reservations ['7b2346c7-cc63-4152-a522-ccbaf4f6717f', '53ffab87-5a97-445f-a53a-6c206b6df7d2', 'fc1ea651-48c8-4b7b-871b-1b109963b71d', 'eb6cc718-d51a-4ea1-82ba-796857dcc7c9'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 20:57:05.913082 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:05.976518 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:06.045186 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:57:06.052552 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Deleted volume successfully. Sep 17 20:57:06.072130 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7499c455-8751-41a7-b92f-a504a4a01bf5 tempest-LiveAutoBlockMigrationV225Test-1505099207 None] Lock "cinder-d2a1c319-7ed9-4fb7-af62-52e9373195bf-delete_volume" released by "delete_volume" :: held 0.639s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 20:57:16.058544 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=38,cinder:UPDATE=15,cinder:INSERT=3,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:21.435262 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=28,cinder:UPDATE=20,cinder:INSERT=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:21.923842 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-2e7dd0df-9590-4df0-a952-7a27496e91b7 req-813af81b-1312-43b0-a7a7-7c7f366b1562 tempest-LiveMigrationTest-703140491 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:57:23.186277 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-2e7dd0df-9590-4df0-a952-7a27496e91b7 req-813af81b-1312-43b0-a7a7-7c7f366b1562 tempest-LiveMigrationTest-703140491 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.262s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:57:23.187972 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-2e7dd0df-9590-4df0-a952-7a27496e91b7 req-813af81b-1312-43b0-a7a7-7c7f366b1562 tempest-LiveMigrationTest-703140491 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-67568702-e073-4ebf-aeae-310160ebcb1e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '67568702-e073-4ebf-aeae-310160ebcb1e', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:57:23.232319 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-2e7dd0df-9590-4df0-a952-7a27496e91b7 req-813af81b-1312-43b0-a7a7-7c7f366b1562 tempest-LiveMigrationTest-703140491 None] Connection info returned from driver {'name': 'volumes/volume-67568702-e073-4ebf-aeae-310160ebcb1e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '67568702-e073-4ebf-aeae-310160ebcb1e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '85347c21-09b2-44ef-84f5-64c3ad1faefc'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:57:23.318833 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-2e7dd0df-9590-4df0-a952-7a27496e91b7 req-813af81b-1312-43b0-a7a7-7c7f366b1562 tempest-LiveMigrationTest-703140491 None] attachment_update completed successfully. Sep 17 20:57:32.244007 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-2e7dd0df-9590-4df0-a952-7a27496e91b7 req-b1e22281-4207-432c-b5b1-7c9d1a2cc6a3 tempest-LiveMigrationTest-703140491 None] Terminate volume connection completed successfully. Sep 17 20:57:32.244734 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-2e7dd0df-9590-4df0-a952-7a27496e91b7 req-b1e22281-4207-432c-b5b1-7c9d1a2cc6a3 tempest-LiveMigrationTest-703140491 None] Deleting attachment 0ac9f7b4-1309-4212-b108-62093f4c19dc. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 20:57:48.488564 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=21,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:48.505103 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-c4af3c25-7daa-4c54-880d-d04dfe500386 tempest-LiveMigrationTest-128259988 None] Terminate volume connection completed successfully. Sep 17 20:57:48.507886 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-c4af3c25-7daa-4c54-880d-d04dfe500386 tempest-LiveMigrationTest-128259988 None] Deleting attachment 85347c21-09b2-44ef-84f5-64c3ad1faefc. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 20:57:48.834578 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] Acquiring lock "cinder-67568702-e073-4ebf-aeae-310160ebcb1e-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 20:57:48.859230 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] Lock "cinder-67568702-e073-4ebf-aeae-310160ebcb1e-delete_volume" acquired by "delete_volume" :: waited 0.025s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 20:57:48.981379 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:49.129809 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 20:57:49.160944 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] deleting rbd volume volume-67568702-e073-4ebf-aeae-310160ebcb1e {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 20:57:50.158997 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 20:57:50.223481 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] deleting parent snapshot volume-67568702-e073-4ebf-aeae-310160ebcb1e.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 20:57:51.708030 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] Acquiring lock "cinder-39a2a629-4cdb-497c-8f9e-729c11354b4b-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 20:57:51.726626 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=26,cinder:UPDATE=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:57:51.800972 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] Lock "cinder-39a2a629-4cdb-497c-8f9e-729c11354b4b-delete_volume" acquired by "delete_volume" :: waited 0.093s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 20:57:51.944123 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] Created reservations ['61b11421-10f9-42f4-99bf-03c32dc1d6a3', '288e12d6-e102-4e7d-ae7f-ca831b1110a0', '8855a50a-9de3-4396-a87c-d8d8770a4ae0', '1c85f4e0-4e5c-489d-96e7-3cd671e35d3b'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 20:57:52.053173 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:52.170450 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 20:57:52.170988 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] Volume volume-39a2a629-4cdb-497c-8f9e-729c11354b4b is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 20:57:52.196525 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] deleting rbd volume volume-39a2a629-4cdb-497c-8f9e-729c11354b4b {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 20:57:52.241600 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:52.335237 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:52.481572 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:57:52.487828 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] Deleted volume successfully. Sep 17 20:57:52.513516 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-2d5a52e4-6f81-43a9-8636-9675f7b2ac6c req-57c71764-5a45-4c81-9611-ab1c89f4d95e tempest-LiveMigrationTest-128259988 None] Lock "cinder-67568702-e073-4ebf-aeae-310160ebcb1e-delete_volume" released by "delete_volume" :: held 3.654s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 20:57:53.187396 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] Created reservations ['b5023594-8af9-4c90-a17c-41cbd843114a', 'ce17ab2f-6cd4-4d17-9a22-75db029dbc54', 'ca0a854f-b7b0-4d09-a2e1-835abb59b977', 'f2a8389f-4841-44c2-be70-5eb25da0a699'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 20:57:53.329653 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:53.432030 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:57:53.501248 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:57:53.506188 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] Deleted volume successfully. Sep 17 20:57:53.525840 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f8925bd5-f984-4283-8a02-fbb6b849090e tempest-LiveMigrationTest-128259988 None] Lock "cinder-39a2a629-4cdb-497c-8f9e-729c11354b4b-delete_volume" released by "delete_volume" :: held 1.725s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 20:58:01.036906 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:58:01.037831 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:58:01.119020 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:58:01.213448 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:58:10.622991 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 20:58:10.668947 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Flow 'volume_create_manager' (b54e81c0-a7eb-4e03-bece-ad4d3fe3adee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:58:10.677965 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (090bf3bc-cd89-4319-86c0-c441bb1f2b06) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:58:10.718471 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (090bf3bc-cd89-4319-86c0-c441bb1f2b06) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T20:58:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-DeleteServersTestJSON-volume-1210876511',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6852ac9c-d055-4788-8316-b59dbb632040,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='89b46ae7347d4f2daf02f627e41a4847',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T20:58:11Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T20:58:11Z,use_quota=True,user_id='3009c5feb1ef43a08b07996d8f63bf11',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:58:10.728193 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (95e1b157-ffcc-4ebb-9878-6d38e0670539) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:58:10.733822 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (95e1b157-ffcc-4ebb-9878-6d38e0670539) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:58:10.740231 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dae95672-eb21-4c30-8c22-0dccef9284aa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:58:10.746073 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dae95672-eb21-4c30-8c22-0dccef9284aa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6852ac9c-d055-4788-8316-b59dbb632040', 'volume_name': 'volume-6852ac9c-d055-4788-8316-b59dbb632040', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:58:10.752836 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9ce500a0-f423-4ffe-915b-730cf12ef447) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:58:10.759295 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9ce500a0-f423-4ffe-915b-730cf12ef447) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:58:10.764253 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2be41206-ee36-4c9b-928c-7a8a824e9301) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:58:10.767018 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Volume 6852ac9c-d055-4788-8316-b59dbb632040: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6852ac9c-d055-4788-8316-b59dbb632040', 'volume_size': 1} Sep 17 20:58:10.767603 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] creating volume 'volume-6852ac9c-d055-4788-8316-b59dbb632040' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 20:58:10.768916 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:58:10.885183 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 20:58:10.885903 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 20:58:10.892540 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2be41206-ee36-4c9b-928c-7a8a824e9301) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6852ac9c-d055-4788-8316-b59dbb632040', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:58:10.906035 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (775e601a-8471-4f76-a93e-0f3db8b0c4bc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 20:58:10.928531 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Volume volume-6852ac9c-d055-4788-8316-b59dbb632040 (6852ac9c-d055-4788-8316-b59dbb632040): created successfully Sep 17 20:58:10.936597 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (775e601a-8471-4f76-a93e-0f3db8b0c4bc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 20:58:10.957402 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Flow 'volume_create_manager' (b54e81c0-a7eb-4e03-bece-ad4d3fe3adee) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 20:58:10.986345 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-78d07416-ca89-4826-94b1-d0acc52cbd32 tempest-DeleteServersTestJSON-583053734 None] Created volume successfully. Sep 17 20:58:12.958381 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-4bb52fa9-e92f-4191-9d64-794c5377f5b3 req-892c4665-c302-4451-883b-ff54ea1961dd tempest-DeleteServersTestJSON-583053734 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 20:58:14.429084 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-4bb52fa9-e92f-4191-9d64-794c5377f5b3 req-892c4665-c302-4451-883b-ff54ea1961dd tempest-DeleteServersTestJSON-583053734 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.472s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 20:58:14.430009 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-4bb52fa9-e92f-4191-9d64-794c5377f5b3 req-892c4665-c302-4451-883b-ff54ea1961dd tempest-DeleteServersTestJSON-583053734 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-6852ac9c-d055-4788-8316-b59dbb632040', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '6852ac9c-d055-4788-8316-b59dbb632040', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 20:58:14.455535 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-4bb52fa9-e92f-4191-9d64-794c5377f5b3 req-892c4665-c302-4451-883b-ff54ea1961dd tempest-DeleteServersTestJSON-583053734 None] Connection info returned from driver {'name': 'volumes/volume-6852ac9c-d055-4788-8316-b59dbb632040', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '6852ac9c-d055-4788-8316-b59dbb632040', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '8bef0a65-ac84-49ec-9d17-fd7e5e986903'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 20:58:14.533822 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-4bb52fa9-e92f-4191-9d64-794c5377f5b3 req-892c4665-c302-4451-883b-ff54ea1961dd tempest-DeleteServersTestJSON-583053734 None] attachment_update completed successfully. Sep 17 20:58:21.701990 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=49,cinder:INSERT=8,cinder:UPDATE=16,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:23.119362 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=34,cinder:UPDATE=25,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:58:23.128125 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-4d3d7a43-c4f7-4516-9d46-2d87c5d3ebcb req-abaab3f5-8197-4e68-9959-1e4c38f74ff9 tempest-DeleteServersTestJSON-583053734 None] Terminate volume connection completed successfully. Sep 17 20:58:23.128976 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-4d3d7a43-c4f7-4516-9d46-2d87c5d3ebcb req-abaab3f5-8197-4e68-9959-1e4c38f74ff9 tempest-DeleteServersTestJSON-583053734 None] Deleting attachment 8bef0a65-ac84-49ec-9d17-fd7e5e986903. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 20:58:51.714743 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:01.040683 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 20:59:01.041695 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:59:01.110299 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 20:59:01.178133 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 20:59:01.709012 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:11.716670 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:31.722884 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 20:59:51.724859 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:01.045270 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:00:01.046009 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:00:01.163634 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:00:01.273263 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:00:11.725876 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:31.725485 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:50.922715 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:00:50.924209 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-98c80d17-bbd8-45f1-a875-518656b85c2a req-435def5b-7534-4142-aab0-7d279707ab53 tempest-TaggedBootDevicesTest-356253908 None] Terminate volume connection completed successfully. Sep 17 21:00:50.924623 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-98c80d17-bbd8-45f1-a875-518656b85c2a req-435def5b-7534-4142-aab0-7d279707ab53 tempest-TaggedBootDevicesTest-356253908 None] Deleting attachment 7eb215eb-8b80-4fc5-a916-86ba7a7c14fb. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:00:51.116092 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-98c80d17-bbd8-45f1-a875-518656b85c2a req-4397908d-8f5e-48f8-982a-91862338b618 tempest-TaggedBootDevicesTest-356253908 None] Terminate volume connection completed successfully. Sep 17 21:00:51.116779 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-98c80d17-bbd8-45f1-a875-518656b85c2a req-4397908d-8f5e-48f8-982a-91862338b618 tempest-TaggedBootDevicesTest-356253908 None] Deleting attachment a88e6abe-3519-4581-b8d1-391709977707. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:00:51.318233 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-98c80d17-bbd8-45f1-a875-518656b85c2a req-40a45285-3847-4cc2-9528-9b98fd325043 tempest-TaggedBootDevicesTest-356253908 None] Terminate volume connection completed successfully. Sep 17 21:00:51.319777 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-98c80d17-bbd8-45f1-a875-518656b85c2a req-40a45285-3847-4cc2-9528-9b98fd325043 tempest-TaggedBootDevicesTest-356253908 None] Deleting attachment 985d11cc-c1ab-48b2-9279-44882378589a. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:01:01.043495 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:01:01.044166 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:01.135816 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:01.194674 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:01:01.317037 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:07.588078 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] Acquiring lock "cinder-1a721e0a-0c66-4cab-9a36-ed78def77377-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:01:07.620525 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] Lock "cinder-1a721e0a-0c66-4cab-9a36-ed78def77377-delete_volume" acquired by "delete_volume" :: waited 0.034s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:01:07.705006 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:07.782810 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:01:07.783269 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] Volume volume-1a721e0a-0c66-4cab-9a36-ed78def77377 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:01:07.797358 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] deleting rbd volume volume-1a721e0a-0c66-4cab-9a36-ed78def77377 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:01:08.117235 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] Created reservations ['70709b66-e64b-4cb4-8eaf-ca19feaf335a', '1e7edb4a-f194-4c42-a55d-1245761dcef0', 'd37825af-707e-47e8-a185-23d81079ec61', 'a8d6884d-f623-4610-a634-3e38f0f2f7ef'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:01:08.211302 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:08.278145 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:08.349499 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:01:08.363117 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] Deleted volume successfully. Sep 17 21:01:08.391053 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d3c8f488-1244-4e49-8ec5-b0079fe03548 tempest-TaggedBootDevicesTest-356253908 None] Lock "cinder-1a721e0a-0c66-4cab-9a36-ed78def77377-delete_volume" released by "delete_volume" :: held 0.771s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:01:08.819585 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] Acquiring lock "cinder-39a25695-6a20-4519-b4cc-233d172fde0a-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:01:08.834602 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] Lock "cinder-39a25695-6a20-4519-b4cc-233d172fde0a-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:01:08.893181 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:09.013826 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:01:09.014254 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] Volume volume-39a25695-6a20-4519-b4cc-233d172fde0a is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:01:09.031492 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] deleting rbd volume volume-39a25695-6a20-4519-b4cc-233d172fde0a {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:01:09.353429 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] Created reservations ['87b425e7-a840-46f5-99c9-c5df89c3644c', '01955f4b-5c84-47dc-a314-25293026a08f', '43e33cd9-2393-47bf-97bc-5ecef1eadac9', '856a118e-7905-4111-8bd9-95cb1a9215c1'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:01:09.417939 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:09.453236 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:09.489907 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:01:09.494349 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] Deleted volume successfully. Sep 17 21:01:09.514480 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a4deb0d4-95bf-4d25-9c11-321af130ca00 tempest-TaggedBootDevicesTest-356253908 None] Lock "cinder-39a25695-6a20-4519-b4cc-233d172fde0a-delete_volume" released by "delete_volume" :: held 0.680s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:01:10.039933 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] Acquiring lock "cinder-99848cf4-d1b1-4750-bd2d-84def7a9a754-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:01:10.075583 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] Lock "cinder-99848cf4-d1b1-4750-bd2d-84def7a9a754-delete_volume" acquired by "delete_volume" :: waited 0.036s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:01:10.132936 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:10.232601 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:01:10.243346 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] deleting rbd volume volume-99848cf4-d1b1-4750-bd2d-84def7a9a754 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:01:10.537692 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:01:10.572328 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] deleting parent snapshot volume-99848cf4-d1b1-4750-bd2d-84def7a9a754.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:01:11.135334 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] Created reservations ['05bd3e53-e554-473e-b775-cb339208ce17', '2e44e14d-9095-4edf-8920-614eca6be72a', 'c629a42d-c244-43df-ae42-86500bae2f61', '717e9324-f20a-468f-8b4f-ea448d4b96e2'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:01:11.245084 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:11.331427 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:11.416273 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:01:11.423108 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] Deleted volume successfully. Sep 17 21:01:11.442343 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ea29ce4b-77cd-4857-8bb2-ba19c3bc0563 tempest-TaggedBootDevicesTest-356253908 None] Lock "cinder-99848cf4-d1b1-4750-bd2d-84def7a9a754-delete_volume" released by "delete_volume" :: held 1.368s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:01:21.429486 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=43,cinder:UPDATE=21,cinder:INSERT=6,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:29.547619 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:01:29.593760 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Flow 'volume_create_manager' (a29302c6-4d1a-44ee-ba9d-7d685b66e58a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:01:29.609814 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6f569df3-85f1-495e-aff9-d37ccd3e516f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:29.629451 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6f569df3-85f1-495e-aff9-d37ccd3e516f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:01:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServersNegativeTestJSON-volume-954320327',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=7e34ad5d-40f7-4099-97d8-4a4e3ba019d8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='36ce01157eec446dbcc93b2948b4be90',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:01:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:01:29Z,use_quota=True,user_id='33fa2d463884464fbe90666c98da0fd7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:29.636232 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (def61237-1801-482a-bc2b-f2f1d821e967) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:29.642629 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (def61237-1801-482a-bc2b-f2f1d821e967) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:29.650968 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8d86f444-fc4f-491b-860d-8db23bd97b91) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:29.664212 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8d86f444-fc4f-491b-860d-8db23bd97b91) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '7e34ad5d-40f7-4099-97d8-4a4e3ba019d8', 'volume_name': 'volume-7e34ad5d-40f7-4099-97d8-4a4e3ba019d8', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:29.669936 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (89b5093f-7cc8-43d1-9dd7-508499f8b672) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:29.676080 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (89b5093f-7cc8-43d1-9dd7-508499f8b672) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:29.683110 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (94d7d185-d4bb-4663-943d-4d4ceabaa1ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:29.686320 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Volume 7e34ad5d-40f7-4099-97d8-4a4e3ba019d8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-7e34ad5d-40f7-4099-97d8-4a4e3ba019d8', 'volume_size': 1} Sep 17 21:01:29.686613 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] creating volume 'volume-7e34ad5d-40f7-4099-97d8-4a4e3ba019d8' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:01:29.694314 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:29.847192 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:01:29.847863 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:01:29.852636 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (94d7d185-d4bb-4663-943d-4d4ceabaa1ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7e34ad5d-40f7-4099-97d8-4a4e3ba019d8', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:29.858127 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e7bbd45e-3242-4be5-a213-e337d70f8646) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:29.872923 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Volume volume-7e34ad5d-40f7-4099-97d8-4a4e3ba019d8 (7e34ad5d-40f7-4099-97d8-4a4e3ba019d8): created successfully Sep 17 21:01:29.883021 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e7bbd45e-3242-4be5-a213-e337d70f8646) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:29.913386 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Flow 'volume_create_manager' (a29302c6-4d1a-44ee-ba9d-7d685b66e58a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:01:29.934579 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ebcaa6dd-ac97-4efe-b7ea-d83b23a0bbcd tempest-ServersNegativeTestJSON-33808428 None] Created volume successfully. Sep 17 21:01:31.748056 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=49,cinder:UPDATE=33,cinder:INSERT=6,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:01:34.396856 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:01:34.443249 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Flow 'volume_create_manager' (53baf669-8786-4a9a-9a56-a7ff9d8c680c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:01:34.450237 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (48ae3874-e703-49aa-8fd4-ed76723b4ec8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:34.473083 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (48ae3874-e703-49aa-8fd4-ed76723b4ec8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:01:34Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServersNegativeTestJSON-volume-2126170626',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='36ce01157eec446dbcc93b2948b4be90',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:01:34Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:01:34Z,use_quota=True,user_id='33fa2d463884464fbe90666c98da0fd7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:34.488720 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (39262f34-4c71-4abf-8f2d-a00627301c27) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:34.500677 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (39262f34-4c71-4abf-8f2d-a00627301c27) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:34.513503 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3eb67936-f2ed-4707-b323-5fbd4f3ac1fa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:34.546303 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3eb67936-f2ed-4707-b323-5fbd4f3ac1fa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f', 'volume_name': 'volume-cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:34.554187 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b833daf1-3e6d-44a4-96d1-f00b4d7be11c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:34.585991 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b833daf1-3e6d-44a4-96d1-f00b4d7be11c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:34.603717 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f6c27899-1e19-476a-a6ad-13a041dce115) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:34.604849 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Volume cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f', 'volume_size': 1} Sep 17 21:01:34.605245 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] creating volume 'volume-cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:01:34.605941 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:34.804882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:01:34.805371 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:01:34.809530 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f6c27899-1e19-476a-a6ad-13a041dce115) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:34.833314 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f802331c-7e90-463b-96ec-e82cf01d19a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:34.878168 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Volume volume-cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f (cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f): created successfully Sep 17 21:01:34.887938 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f802331c-7e90-463b-96ec-e82cf01d19a8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:34.916294 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Flow 'volume_create_manager' (53baf669-8786-4a9a-9a56-a7ff9d8c680c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:01:34.943663 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-12cc637f-cf21-40a0-b756-b998b7c2b843 tempest-ServersNegativeTestJSON-33808428 None] Created volume successfully. Sep 17 21:01:36.485559 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:01:36.559402 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Flow 'volume_create_manager' (d350be1d-a05f-4ba3-ade3-71faeaaf1e30) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:01:36.569682 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (495cd6f8-5e11-4dfe-91b2-ecf3f76f19a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:36.589650 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (495cd6f8-5e11-4dfe-91b2-ecf3f76f19a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:01:36Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachSCSIVolumeTestJSON-volume-1775558017',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=25a02b63-f7d3-40d4-8373-cc437ceef99e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f8813aaee4604882b4cd7a2a8df58077',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:01:36Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:01:36Z,use_quota=True,user_id='da5d41f7e95d44d3898f6306bb093ed7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:36.613036 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (98dea1a9-8ca3-4c78-b6a1-3f37e8dd4b18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:36.617268 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (98dea1a9-8ca3-4c78-b6a1-3f37e8dd4b18) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:36.642901 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a06abaa8-79f7-4238-88f5-81d668455eaa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:36.657674 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a06abaa8-79f7-4238-88f5-81d668455eaa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '25a02b63-f7d3-40d4-8373-cc437ceef99e', 'volume_name': 'volume-25a02b63-f7d3-40d4-8373-cc437ceef99e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:36.664388 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f612c2ed-0151-4b99-812b-2a7f25cd75dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:36.682723 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f612c2ed-0151-4b99-812b-2a7f25cd75dd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:36.693507 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (16f4707f-ea0d-4a77-a819-be4f22e75be8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:36.699155 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Volume 25a02b63-f7d3-40d4-8373-cc437ceef99e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-25a02b63-f7d3-40d4-8373-cc437ceef99e', 'volume_size': 1} Sep 17 21:01:36.699563 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] creating volume 'volume-25a02b63-f7d3-40d4-8373-cc437ceef99e' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:01:36.702051 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:01:36.901215 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:01:36.902004 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:01:36.909405 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (16f4707f-ea0d-4a77-a819-be4f22e75be8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-25a02b63-f7d3-40d4-8373-cc437ceef99e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:36.922511 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cd3c40b1-d447-4d95-a9c7-83fe0424ae9e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:01:36.955342 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Volume volume-25a02b63-f7d3-40d4-8373-cc437ceef99e (25a02b63-f7d3-40d4-8373-cc437ceef99e): created successfully Sep 17 21:01:36.961000 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cd3c40b1-d447-4d95-a9c7-83fe0424ae9e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:01:36.972183 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Flow 'volume_create_manager' (d350be1d-a05f-4ba3-ade3-71faeaaf1e30) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:01:36.996438 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f9fc23af-dbcc-4915-9839-34014a8e00f3 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Created volume successfully. Sep 17 21:01:39.422999 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-659ea23f-7956-4674-9729-e3d2f6ea6437 req-7c7e162c-318e-442d-8c87-335593de2b82 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:01:40.998792 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-659ea23f-7956-4674-9729-e3d2f6ea6437 req-7c7e162c-318e-442d-8c87-335593de2b82 tempest-AttachSCSIVolumeTestJSON-2060044451 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.576s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:01:40.999718 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-659ea23f-7956-4674-9729-e3d2f6ea6437 req-7c7e162c-318e-442d-8c87-335593de2b82 tempest-AttachSCSIVolumeTestJSON-2060044451 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-25a02b63-f7d3-40d4-8373-cc437ceef99e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '25a02b63-f7d3-40d4-8373-cc437ceef99e', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:01:41.047181 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-659ea23f-7956-4674-9729-e3d2f6ea6437 req-7c7e162c-318e-442d-8c87-335593de2b82 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Connection info returned from driver {'name': 'volumes/volume-25a02b63-f7d3-40d4-8373-cc437ceef99e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '25a02b63-f7d3-40d4-8373-cc437ceef99e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'd912d40e-646e-4139-befc-ce1df7e39983'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:01:41.124271 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-659ea23f-7956-4674-9729-e3d2f6ea6437 req-7c7e162c-318e-442d-8c87-335593de2b82 tempest-AttachSCSIVolumeTestJSON-2060044451 None] attachment_update completed successfully. Sep 17 21:01:45.351260 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-8af78b65-f12b-4283-8e24-a73f3088ab7c req-97e07772-7088-4a7f-b137-2ab70a097dc6 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Terminate volume connection completed successfully. Sep 17 21:01:45.351866 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-8af78b65-f12b-4283-8e24-a73f3088ab7c req-97e07772-7088-4a7f-b137-2ab70a097dc6 tempest-AttachSCSIVolumeTestJSON-2060044451 None] Deleting attachment d912d40e-646e-4139-befc-ce1df7e39983. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:01:51.747503 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=29,cinder:UPDATE=8,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:01.040264 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:02:01.041871 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:01.134630 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:01.211581 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:02:01.755812 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=26,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:01.851899 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] Acquiring lock "cinder-25a02b63-f7d3-40d4-8373-cc437ceef99e-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:02:01.867724 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] Lock "cinder-25a02b63-f7d3-40d4-8373-cc437ceef99e-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:02:01.957285 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:02.046621 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:02:02.047281 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] Volume volume-25a02b63-f7d3-40d4-8373-cc437ceef99e is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:02:02.060950 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] deleting rbd volume volume-25a02b63-f7d3-40d4-8373-cc437ceef99e {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:02:02.476068 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] Created reservations ['1aafc626-b7c5-4cb0-846a-a6c7adcd264d', 'a3c9624a-953c-4ab7-982f-727e51f8590f', '572ab3e9-4560-4bae-aa5a-9bdd8cdfd82f', '7dc8696b-4dfd-4908-88f3-3ef2750a267c'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:02:02.590766 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:02.660723 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:02.741258 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:02:02.749263 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] Deleted volume successfully. Sep 17 21:02:02.780680 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8c858d50-3a34-4330-85e8-72d9ddc7e49b tempest-AttachSCSIVolumeTestJSON-2060044451 None] Lock "cinder-25a02b63-f7d3-40d4-8373-cc437ceef99e-delete_volume" released by "delete_volume" :: held 0.913s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:02:12.030700 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] Acquiring lock "cinder-6852ac9c-d055-4788-8316-b59dbb632040-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:02:12.046354 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] Lock "cinder-6852ac9c-d055-4788-8316-b59dbb632040-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:02:12.112251 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:12.216335 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:02:12.216851 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] Volume volume-6852ac9c-d055-4788-8316-b59dbb632040 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:02:12.230098 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] deleting rbd volume volume-6852ac9c-d055-4788-8316-b59dbb632040 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:02:12.927044 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] Created reservations ['93463792-988e-4b71-9718-098957e0d099', '47cce8b1-338e-471e-9e04-5023eb3d724f', 'af980890-d49a-44ce-bff6-47dc095bc756', 'a1a148fb-ad0c-4790-9b67-093bd1315837'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:02:13.063406 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:13.132824 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:13.187903 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:02:13.192873 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] Deleted volume successfully. Sep 17 21:02:13.221142 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-55b757e2-cfe1-40d4-9505-e0b094db92ab tempest-DeleteServersTestJSON-583053734 None] Lock "cinder-6852ac9c-d055-4788-8316-b59dbb632040-delete_volume" released by "delete_volume" :: held 1.175s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:02:13.641105 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:02:13.716893 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Flow 'volume_create_manager' (43285a84-ed3f-4002-bfce-9d999b33d55a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:02:13.730578 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2218575e-8a26-468d-88a2-3f8e18bc35cd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:02:13.771394 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2218575e-8a26-468d-88a2-3f8e18bc35cd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:02:13Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTestJSON-Volume-1499371743',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f9d7ea0f-64db-480f-b2ac-a18ec144ee2b,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='2b777025c44a4535bea435d55f8cb1e3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:02:14Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:02:14Z,use_quota=True,user_id='fe663342df9f495d8203d8bb817868ec',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:02:13.792883 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8c043afb-08a8-4c4f-95ec-4a24e5a265df) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:02:13.805796 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8c043afb-08a8-4c4f-95ec-4a24e5a265df) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:02:13.814089 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (108790b8-1ecb-4d71-a9d6-54f1e8857191) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:02:13.825810 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (108790b8-1ecb-4d71-a9d6-54f1e8857191) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f9d7ea0f-64db-480f-b2ac-a18ec144ee2b', 'volume_name': 'volume-f9d7ea0f-64db-480f-b2ac-a18ec144ee2b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:02:13.833366 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7e0ae408-75c1-4bd0-8035-00197749443a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:02:13.840390 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7e0ae408-75c1-4bd0-8035-00197749443a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:02:13.852213 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6118c1f3-b6ef-45ba-9998-8b89fa2d47c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:02:13.854114 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Volume f9d7ea0f-64db-480f-b2ac-a18ec144ee2b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f9d7ea0f-64db-480f-b2ac-a18ec144ee2b', 'volume_size': 1} Sep 17 21:02:13.854620 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] creating volume 'volume-f9d7ea0f-64db-480f-b2ac-a18ec144ee2b' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:02:13.855569 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:14.026403 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:02:14.026798 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:02:14.031273 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6118c1f3-b6ef-45ba-9998-8b89fa2d47c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f9d7ea0f-64db-480f-b2ac-a18ec144ee2b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:02:14.041148 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (49e07ccb-327b-409f-8d6f-12c21c30ca16) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:02:14.100690 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Volume volume-f9d7ea0f-64db-480f-b2ac-a18ec144ee2b (f9d7ea0f-64db-480f-b2ac-a18ec144ee2b): created successfully Sep 17 21:02:14.112622 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (49e07ccb-327b-409f-8d6f-12c21c30ca16) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:02:14.128921 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Flow 'volume_create_manager' (43285a84-ed3f-4002-bfce-9d999b33d55a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:02:14.143470 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-33b71ced-2d46-49dd-8691-fad95ac57b95 req-4962887b-6dd9-4dd6-9b9e-77434524b46a tempest-VolumesGetTestJSON-1283102623 None] Created volume successfully. Sep 17 21:02:15.094313 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] Acquiring lock "cinder-f9d7ea0f-64db-480f-b2ac-a18ec144ee2b-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:02:15.127218 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] Lock "cinder-f9d7ea0f-64db-480f-b2ac-a18ec144ee2b-delete_volume" acquired by "delete_volume" :: waited 0.033s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:02:15.203615 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:15.331418 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:02:15.331907 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] Volume volume-f9d7ea0f-64db-480f-b2ac-a18ec144ee2b is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:02:15.348689 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] deleting rbd volume volume-f9d7ea0f-64db-480f-b2ac-a18ec144ee2b {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:02:15.817520 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] Created reservations ['9a61527b-7268-4c08-bc0d-14b26a63c65e', '99f8e0da-f2d2-4831-b166-6a7fe188ae02', '02a91143-be67-47e6-8c69-2594e1a7f893', '26bcdbd0-ee4c-4864-b0d6-d1e31295731f'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:02:15.939052 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:16.041683 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:02:16.128007 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:02:16.136430 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] Deleted volume successfully. Sep 17 21:02:16.162344 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-3d33a3cb-61d5-4ed3-9ee8-df45cb70f2a4 req-6c3ff7ed-22a6-4326-bc24-6cafc4033136 tempest-VolumesGetTestJSON-1283102623 None] Lock "cinder-f9d7ea0f-64db-480f-b2ac-a18ec144ee2b-delete_volume" released by "delete_volume" :: held 1.035s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:02:21.775936 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=40,cinder:UPDATE=22,cinder:INSERT=4,cinder:DELETE=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:31.777981 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=49,cinder:INSERT=8,cinder:UPDATE=32 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:02:51.756487 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:01.043206 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:03:01.044601 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:03:01.114516 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:03:01.194252 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:03:11.746761 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:03:31.744355 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:01.046653 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:04:01.048098 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:04:01.229254 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:04:01.324186 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:04:01.747377 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:11.744298 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:31.746083 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:04:51.754666 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:01.047612 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:05:01.048985 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:05:01.137923 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:05:01.236761 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:05:10.007487 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] Acquiring lock "cinder-cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:05:10.042866 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] Lock "cinder-cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f-delete_volume" acquired by "delete_volume" :: waited 0.035s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:05:10.138939 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:05:10.240376 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:05:10.240867 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] Volume volume-cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:05:10.256295 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] deleting rbd volume volume-cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:05:10.698921 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] Created reservations ['cefd344c-a1ff-4fa8-b197-dd388eb1c3da', '70e32732-f3ee-42e2-8511-dc762d8332f9', '4d06921d-ac8d-452d-b2ac-c2c1a5f22468', '17935178-12cf-4f6c-b324-484fda903836'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:05:10.761234 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:05:10.846330 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:05:10.934586 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:05:10.942083 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] Deleted volume successfully. Sep 17 21:05:10.967113 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ae498228-b26a-4024-8d6f-56d0a37b9d0c tempest-ServersNegativeTestJSON-33808428 None] Lock "cinder-cca364ab-72d2-4b09-8ddb-85f1ec1d6b4f-delete_volume" released by "delete_volume" :: held 0.925s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:05:11.257485 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] Acquiring lock "cinder-7e34ad5d-40f7-4099-97d8-4a4e3ba019d8-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:05:11.289679 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] Lock "cinder-7e34ad5d-40f7-4099-97d8-4a4e3ba019d8-delete_volume" acquired by "delete_volume" :: waited 0.032s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:05:11.370546 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:05:11.478131 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:05:11.478906 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] Volume volume-7e34ad5d-40f7-4099-97d8-4a4e3ba019d8 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:05:11.491485 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] deleting rbd volume volume-7e34ad5d-40f7-4099-97d8-4a4e3ba019d8 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:05:11.908108 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=20,cinder:UPDATE=11,cinder:INSERT=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:11.955562 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] Created reservations ['70c83813-a180-4546-8f3d-5750f67255e0', 'fd7a3427-6fed-47c4-9bbc-3488885a00fd', 'ff2ad8c7-1cc9-4208-b07e-f6e98ebde6e5', '520d87a0-5ecc-4463-82b4-0149ac0555f2'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:05:12.065255 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:05:12.172657 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:05:12.273418 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:05:12.295297 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] Deleted volume successfully. Sep 17 21:05:12.320921 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a6d96207-c866-45f1-b70e-4d83e7c15214 tempest-ServersNegativeTestJSON-33808428 None] Lock "cinder-7e34ad5d-40f7-4099-97d8-4a4e3ba019d8-delete_volume" released by "delete_volume" :: held 1.031s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:05:31.750369 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=39,cinder:UPDATE=18,cinder:INSERT=3,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:05:51.743464 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:INSERT=3,cinder:UPDATE=8,cinder:SELECT=7,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:01.043530 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:06:01.047712 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:01.098519 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:01.165368 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:06:01.198371 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:06:01.248926 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (57ef17b8-b3b2-4c8e-a873-02e8513ba387) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:01.260101 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5971f1fb-5c9c-453f-8caf-5cd99e61202e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:01.284233 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5971f1fb-5c9c-453f-8caf-5cd99e61202e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:06:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-539421444',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3eb72a32-7071-4518-8571-6d22f73e4404,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='598139a6b6764e80822b82231f572770',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:06:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:06:01Z,use_quota=True,user_id='1933f2aed04a45a28bb2617337faa719',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:01.291582 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4dabbc26-48ce-4c23-a986-b57d4152a7d2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:01.297756 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4dabbc26-48ce-4c23-a986-b57d4152a7d2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:01.305574 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (9feb8e05-4f41-445a-8640-6d7558df8bf6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:01.311210 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (9feb8e05-4f41-445a-8640-6d7558df8bf6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '3eb72a32-7071-4518-8571-6d22f73e4404', 'volume_name': 'volume-3eb72a32-7071-4518-8571-6d22f73e4404', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:01.318032 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e22df977-5722-4258-9059-d4d6fa273bc0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:01.322982 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e22df977-5722-4258-9059-d4d6fa273bc0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:01.330642 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a9992896-a2a7-4fb6-a0ca-d65f137c09e9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:01.334047 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Volume 3eb72a32-7071-4518-8571-6d22f73e4404: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-3eb72a32-7071-4518-8571-6d22f73e4404', 'volume_size': 1} Sep 17 21:06:01.334878 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] creating volume 'volume-3eb72a32-7071-4518-8571-6d22f73e4404' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:06:01.336092 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:01.431207 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:06:01.431580 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:06:01.436062 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a9992896-a2a7-4fb6-a0ca-d65f137c09e9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3eb72a32-7071-4518-8571-6d22f73e4404', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:01.442786 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9c3c384f-8dc6-45cd-a380-7be2bc328e5c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:01.458004 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Volume volume-3eb72a32-7071-4518-8571-6d22f73e4404 (3eb72a32-7071-4518-8571-6d22f73e4404): created successfully Sep 17 21:06:01.488863 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9c3c384f-8dc6-45cd-a380-7be2bc328e5c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:01.511275 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (57ef17b8-b3b2-4c8e-a873-02e8513ba387) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:01.554423 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a181f87e-608c-4616-8fbf-f29f28e1100f tempest-VolumesActionsTest-176422482 None] Created volume successfully. Sep 17 21:06:02.328437 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ed138aca-4c6d-4930-8f68-39eaca21e62c tempest-VolumesActionsTest-176422482 None] Acquiring lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404" by "attach_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:06:02.362882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ed138aca-4c6d-4930-8f68-39eaca21e62c tempest-VolumesActionsTest-176422482 None] Lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404" acquired by "attach_volume" :: waited 0.033s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:06:02.412625 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=10,cinder:UPDATE=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:02.470183 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ed138aca-4c6d-4930-8f68-39eaca21e62c tempest-VolumesActionsTest-176422482 None] Attaching volume 3eb72a32-7071-4518-8571-6d22f73e4404 to instance 829770cb-178d-439d-9efd-ac18ac670932 at mountpoint /dev/sdb on host None. Sep 17 21:06:02.572020 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ed138aca-4c6d-4930-8f68-39eaca21e62c tempest-VolumesActionsTest-176422482 None] Attach volume completed successfully. Sep 17 21:06:02.580584 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ed138aca-4c6d-4930-8f68-39eaca21e62c tempest-VolumesActionsTest-176422482 None] Lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404" released by "attach_volume" :: held 0.219s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:06:03.439742 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b35304ae-da2d-4fe3-99da-232c8df9e14d tempest-VolumesActionsTest-344800113 None] Terminate volume connection completed successfully. Sep 17 21:06:03.469235 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b35304ae-da2d-4fe3-99da-232c8df9e14d tempest-VolumesActionsTest-344800113 None] Acquiring lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404-detach_volume" by "detach_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:06:03.481946 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b35304ae-da2d-4fe3-99da-232c8df9e14d tempest-VolumesActionsTest-344800113 None] Lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404-detach_volume" acquired by "detach_volume" :: waited 0.013s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:06:03.540601 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b35304ae-da2d-4fe3-99da-232c8df9e14d tempest-VolumesActionsTest-344800113 None] Detaching volume 3eb72a32-7071-4518-8571-6d22f73e4404 from instance 829770cb-178d-439d-9efd-ac18ac670932. Sep 17 21:06:03.645641 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b35304ae-da2d-4fe3-99da-232c8df9e14d tempest-VolumesActionsTest-344800113 None] Detach volume completed successfully. Sep 17 21:06:03.652549 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b35304ae-da2d-4fe3-99da-232c8df9e14d tempest-VolumesActionsTest-344800113 None] Lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404-detach_volume" released by "detach_volume" :: held 0.171s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:06:03.922548 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-0e65b05f-795a-476b-bba3-9ff4ab7a164b tempest-VolumesActionsTest-176422482 None] Acquiring lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404-detach_volume" by "detach_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:06:03.935624 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-0e65b05f-795a-476b-bba3-9ff4ab7a164b tempest-VolumesActionsTest-176422482 None] Lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404-detach_volume" acquired by "detach_volume" :: waited 0.014s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:06:03.936456 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-0e65b05f-795a-476b-bba3-9ff4ab7a164b tempest-VolumesActionsTest-176422482 None] Volume detach called, but volume not attached. Sep 17 21:06:03.962036 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-0e65b05f-795a-476b-bba3-9ff4ab7a164b tempest-VolumesActionsTest-176422482 None] Lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404-detach_volume" released by "detach_volume" :: held 0.026s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:06:12.067116 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:06:12.090951 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (8637e950-0137-4088-a683-2c1677147ec5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:12.098745 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c65714be-4686-414e-9d19-22fdb8dbc09f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:12.121882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c65714be-4686-414e-9d19-22fdb8dbc09f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:06:12Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-107932361',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=05df4144-acdc-4fb7-b0a4-246502e84f58,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='598139a6b6764e80822b82231f572770',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:06:12Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:06:12Z,use_quota=True,user_id='1933f2aed04a45a28bb2617337faa719',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:12.128248 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9326f31e-3af1-4a50-9adb-92b3e032383c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:12.133273 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9326f31e-3af1-4a50-9adb-92b3e032383c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:12.139066 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a9210ba8-2763-4a02-982b-eee8b1d3779c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:12.143583 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a9210ba8-2763-4a02-982b-eee8b1d3779c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '05df4144-acdc-4fb7-b0a4-246502e84f58', 'volume_name': 'volume-05df4144-acdc-4fb7-b0a4-246502e84f58', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:12.148546 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2bbec09c-5c8d-4821-ae58-e03ce4e81f3d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:12.152721 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2bbec09c-5c8d-4821-ae58-e03ce4e81f3d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:12.157603 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b1994178-3aac-42cd-a396-fa2ee9c946c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:12.158711 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Volume 05df4144-acdc-4fb7-b0a4-246502e84f58: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-05df4144-acdc-4fb7-b0a4-246502e84f58', 'volume_size': 1} Sep 17 21:06:12.159116 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] creating volume 'volume-05df4144-acdc-4fb7-b0a4-246502e84f58' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:06:12.159894 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:12.251633 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:06:12.252018 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:06:12.256441 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b1994178-3aac-42cd-a396-fa2ee9c946c7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-05df4144-acdc-4fb7-b0a4-246502e84f58', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:12.263243 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ec0f05c8-dd2a-4dc1-ad90-3e04d06ba4e6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:12.277837 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Volume volume-05df4144-acdc-4fb7-b0a4-246502e84f58 (05df4144-acdc-4fb7-b0a4-246502e84f58): created successfully Sep 17 21:06:12.283216 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ec0f05c8-dd2a-4dc1-ad90-3e04d06ba4e6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:12.289581 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (8637e950-0137-4088-a683-2c1677147ec5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:12.305535 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f8d2afab-7ab1-431d-8af5-0178201386a8 tempest-VolumesActionsTest-176422482 None] Created volume successfully. Sep 17 21:06:13.327455 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] Acquiring lock "cinder-05df4144-acdc-4fb7-b0a4-246502e84f58-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:06:13.349280 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] Lock "cinder-05df4144-acdc-4fb7-b0a4-246502e84f58-delete_volume" acquired by "delete_volume" :: waited 0.023s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:06:13.402611 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:13.484751 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:06:13.485502 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] Volume volume-05df4144-acdc-4fb7-b0a4-246502e84f58 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:06:13.495200 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] deleting rbd volume volume-05df4144-acdc-4fb7-b0a4-246502e84f58 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:06:13.946503 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] Created reservations ['d72e63f6-2f3d-43d4-ac17-88d3798e3c7e', 'f9c368a7-d8fd-48d8-b643-bf1192f6e9d2', '17312b7c-9ec3-4217-8e84-19b4e49cdbf8', '898f77c9-c3fc-4df4-a353-6dee5d93ba18'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:06:14.024550 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:14.079775 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:14.116900 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:06:14.125851 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] Deleted volume successfully. Sep 17 21:06:14.155436 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3907870e-e4d4-4b71-af0f-8d8f1eb143ef tempest-VolumesActionsTest-344800113 None] Lock "cinder-05df4144-acdc-4fb7-b0a4-246502e84f58-delete_volume" released by "delete_volume" :: held 0.806s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:06:14.874055 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:06:14.895161 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (394fd757-a09a-4126-881b-2bf4652b2b7f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:14.901654 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4f347e73-c9f1-43ef-8597-b76fd77fe35c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:14.919652 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4f347e73-c9f1-43ef-8597-b76fd77fe35c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:06:15Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1261093559',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5caf2cee-de97-4955-a6c9-837206f0900e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='598139a6b6764e80822b82231f572770',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:06:15Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:06:15Z,use_quota=True,user_id='1933f2aed04a45a28bb2617337faa719',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:14.925267 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c132c950-f8d2-4d83-8aad-4bc72ea87dcb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:14.929749 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c132c950-f8d2-4d83-8aad-4bc72ea87dcb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:14.935028 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d8119461-2f72-47d5-9dd4-03cb77382a83) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:14.939636 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d8119461-2f72-47d5-9dd4-03cb77382a83) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5caf2cee-de97-4955-a6c9-837206f0900e', 'volume_name': 'volume-5caf2cee-de97-4955-a6c9-837206f0900e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:14.944391 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (db3b81d8-c8c7-4c42-80f9-86ca52facd96) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:14.948783 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (db3b81d8-c8c7-4c42-80f9-86ca52facd96) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:14.953508 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7f1a0204-f001-4fd7-8fad-11ed96ad7795) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:14.954432 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Volume 5caf2cee-de97-4955-a6c9-837206f0900e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5caf2cee-de97-4955-a6c9-837206f0900e', 'volume_size': 1} Sep 17 21:06:14.954977 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] creating volume 'volume-5caf2cee-de97-4955-a6c9-837206f0900e' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:06:14.955790 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:15.114440 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:06:15.114813 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:06:15.118727 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7f1a0204-f001-4fd7-8fad-11ed96ad7795) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5caf2cee-de97-4955-a6c9-837206f0900e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:15.123165 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (46c6cb3b-de73-450d-b1f9-6b3df9b3086a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:15.153030 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Volume volume-5caf2cee-de97-4955-a6c9-837206f0900e (5caf2cee-de97-4955-a6c9-837206f0900e): created successfully Sep 17 21:06:15.167909 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (46c6cb3b-de73-450d-b1f9-6b3df9b3086a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:15.183187 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (394fd757-a09a-4126-881b-2bf4652b2b7f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:15.210275 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-fc29b014-b826-4c38-b194-c7d17e85d61e tempest-VolumesActionsTest-176422482 None] Created volume successfully. Sep 17 21:06:16.128810 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] Acquiring lock "cinder-5caf2cee-de97-4955-a6c9-837206f0900e-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:06:16.143808 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] Lock "cinder-5caf2cee-de97-4955-a6c9-837206f0900e-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:06:16.194807 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:16.278458 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:06:16.279021 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] Volume volume-5caf2cee-de97-4955-a6c9-837206f0900e is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:06:16.296665 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] deleting rbd volume volume-5caf2cee-de97-4955-a6c9-837206f0900e {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:06:16.820228 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] Created reservations ['dd24ca82-66be-4b16-9502-a9f100f48314', 'e30e6e94-4178-4187-a000-400144da8c9e', '7c3c0405-e96d-4230-9508-d7dcdf91a677', '95667a34-938d-4bb6-8486-b40c6f9adb0d'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:06:16.890021 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:16.982977 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:17.057792 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:06:17.066045 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] Deleted volume successfully. Sep 17 21:06:17.088926 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a9978402-d2c1-4988-a7d4-6e7e287af411 tempest-VolumesActionsTest-344800113 None] Lock "cinder-5caf2cee-de97-4955-a6c9-837206f0900e-delete_volume" released by "delete_volume" :: held 0.945s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:06:17.780960 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:06:17.815865 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (1829bdc9-a2d8-407c-b332-a088b17365a6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:17.822566 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c7ec2fad-9c38-40d2-bede-784ce93924c1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:17.857946 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c7ec2fad-9c38-40d2-bede-784ce93924c1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:06:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1106904867',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a590c5b1-1147-4402-8f8f-91b9841ecf25,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='598139a6b6764e80822b82231f572770',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:06:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:06:18Z,use_quota=True,user_id='1933f2aed04a45a28bb2617337faa719',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:17.866160 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (662bb3e5-dbe2-4ea5-8c71-8dbba402a761) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:17.881823 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (662bb3e5-dbe2-4ea5-8c71-8dbba402a761) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:17.925577 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d17ce7bc-7e04-4c84-9072-2793a32f0fa5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:17.931178 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d17ce7bc-7e04-4c84-9072-2793a32f0fa5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a590c5b1-1147-4402-8f8f-91b9841ecf25', 'volume_name': 'volume-a590c5b1-1147-4402-8f8f-91b9841ecf25', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:17.939582 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a6d2bfb5-1f34-4d53-a696-1169a2869dad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:17.944215 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a6d2bfb5-1f34-4d53-a696-1169a2869dad) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:17.948848 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b66b4b75-8f88-4656-95db-27f7790f996a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:17.949967 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Volume a590c5b1-1147-4402-8f8f-91b9841ecf25: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a590c5b1-1147-4402-8f8f-91b9841ecf25', 'volume_size': 1} Sep 17 21:06:17.950361 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] creating volume 'volume-a590c5b1-1147-4402-8f8f-91b9841ecf25' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:06:17.951138 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:18.065535 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:06:18.065809 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:06:18.070022 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b66b4b75-8f88-4656-95db-27f7790f996a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a590c5b1-1147-4402-8f8f-91b9841ecf25', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:18.074879 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d25a7c70-2fd0-463b-9317-dc3ec19fb09e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:18.097123 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Volume volume-a590c5b1-1147-4402-8f8f-91b9841ecf25 (a590c5b1-1147-4402-8f8f-91b9841ecf25): created successfully Sep 17 21:06:18.103782 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d25a7c70-2fd0-463b-9317-dc3ec19fb09e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:18.113947 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (1829bdc9-a2d8-407c-b332-a088b17365a6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:18.128976 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-4718dc9d-2173-4165-8c05-879ecb6e7a96 tempest-VolumesActionsTest-176422482 None] Created volume successfully. Sep 17 21:06:18.966335 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] Acquiring lock "cinder-a590c5b1-1147-4402-8f8f-91b9841ecf25-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:06:18.984665 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] Lock "cinder-a590c5b1-1147-4402-8f8f-91b9841ecf25-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:06:19.036050 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:19.088404 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:06:19.088986 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] Volume volume-a590c5b1-1147-4402-8f8f-91b9841ecf25 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:06:19.097053 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] deleting rbd volume volume-a590c5b1-1147-4402-8f8f-91b9841ecf25 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:06:19.370949 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] Created reservations ['7f07a992-0a1d-4bea-887f-5859f5700109', '9acb5d07-1c6e-4df6-86de-a50986888257', '53c94b56-4405-47e1-9781-01924620f328', '0e808d7c-3a50-4c50-a918-2f14ab03dd62'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:06:19.375301 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=64,cinder:INSERT=6,cinder:UPDATE=28,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:19.417929 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=56,cinder:UPDATE=32,cinder:DELETE=4,cinder:INSERT=8 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:19.441073 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:19.496327 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:19.535169 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:06:19.545851 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] Deleted volume successfully. Sep 17 21:06:19.562009 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-4b80b6a0-1c5e-4ef3-afde-fd13d30dc239 tempest-VolumesActionsTest-344800113 None] Lock "cinder-a590c5b1-1147-4402-8f8f-91b9841ecf25-delete_volume" released by "delete_volume" :: held 0.577s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:06:20.567040 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:06:20.588481 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (69cbabee-8334-4d18-8b33-840810bb8114) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:20.595145 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (005a5b52-08e5-4fbf-908f-674d1d7aad6e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:20.613193 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (005a5b52-08e5-4fbf-908f-674d1d7aad6e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:06:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-2126829310',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=51567080-ab4e-48f1-babb-221fd7904965,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='598139a6b6764e80822b82231f572770',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:06:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:06:21Z,use_quota=True,user_id='1933f2aed04a45a28bb2617337faa719',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:20.620833 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5426c8ce-d1b4-4313-aa58-4f6587a86e87) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:20.628449 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5426c8ce-d1b4-4313-aa58-4f6587a86e87) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:20.637064 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (20f21335-8cbf-42f1-a1d2-86808f4dc903) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:20.642130 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (20f21335-8cbf-42f1-a1d2-86808f4dc903) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '51567080-ab4e-48f1-babb-221fd7904965', 'volume_name': 'volume-51567080-ab4e-48f1-babb-221fd7904965', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:20.647874 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6919a8c8-e6cc-40a7-9071-f9c6b8a0c51e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:20.651765 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6919a8c8-e6cc-40a7-9071-f9c6b8a0c51e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:20.656285 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0f87d7ff-ede0-4e8b-a049-5882bf1431c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:20.657551 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Volume 51567080-ab4e-48f1-babb-221fd7904965: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-51567080-ab4e-48f1-babb-221fd7904965', 'volume_size': 1} Sep 17 21:06:20.658131 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] creating volume 'volume-51567080-ab4e-48f1-babb-221fd7904965' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:06:20.659193 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:20.807546 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:06:20.808039 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:06:20.812423 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0f87d7ff-ede0-4e8b-a049-5882bf1431c3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-51567080-ab4e-48f1-babb-221fd7904965', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:20.820314 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (69dc646a-578a-4c81-a74c-2d5990fdb48d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:20.833040 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Volume volume-51567080-ab4e-48f1-babb-221fd7904965 (51567080-ab4e-48f1-babb-221fd7904965): created successfully Sep 17 21:06:20.840066 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (69dc646a-578a-4c81-a74c-2d5990fdb48d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:20.847158 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (69cbabee-8334-4d18-8b33-840810bb8114) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:20.859841 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5c25b6cc-9eba-4fbc-97c7-4468bd758495 tempest-VolumesActionsTest-176422482 None] Created volume successfully. Sep 17 21:06:21.853664 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] Acquiring lock "cinder-51567080-ab4e-48f1-babb-221fd7904965-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:06:21.865313 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] Lock "cinder-51567080-ab4e-48f1-babb-221fd7904965-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:06:21.907137 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:22.005838 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:06:22.008433 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] Volume volume-51567080-ab4e-48f1-babb-221fd7904965 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:06:22.042620 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] deleting rbd volume volume-51567080-ab4e-48f1-babb-221fd7904965 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:06:22.437958 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] Created reservations ['d30307bb-0a66-4687-a46e-54a167e28b7f', '73299119-4c39-41f8-aef6-dc88458bbb5c', 'c4025792-4120-4e8e-bf61-ec359f7b9629', '3df60518-eb7f-4711-8598-1ba3f26872d5'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:06:22.508985 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:22.582483 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:22.656591 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:06:22.661520 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] Deleted volume successfully. Sep 17 21:06:22.684054 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f825eac1-8389-4e8b-8c2c-7d12936d4bbd tempest-VolumesActionsTest-344800113 None] Lock "cinder-51567080-ab4e-48f1-babb-221fd7904965-delete_volume" released by "delete_volume" :: held 0.819s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:06:23.465943 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:06:23.490532 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (91b976ff-7153-4039-b834-d3cbe23e4d19) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:23.496884 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2bfb0136-c122-430a-8c4f-3a9d05ab4ab9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:23.516590 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2bfb0136-c122-430a-8c4f-3a9d05ab4ab9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:06:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-325876338',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='598139a6b6764e80822b82231f572770',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:06:23Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:06:23Z,use_quota=True,user_id='1933f2aed04a45a28bb2617337faa719',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:23.522812 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2f58bd67-38d4-4fd6-8ab1-2903ae0ee9bd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:23.526862 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2f58bd67-38d4-4fd6-8ab1-2903ae0ee9bd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:23.532449 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4a4fb3e6-c743-4a06-9a30-349e20722f97) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:23.537358 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4a4fb3e6-c743-4a06-9a30-349e20722f97) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41', 'volume_name': 'volume-655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:23.543109 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e8f7b7ee-fb45-4ec1-ac2a-cedaeed8f43a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:23.575004 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e8f7b7ee-fb45-4ec1-ac2a-cedaeed8f43a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:23.583382 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9159d04b-59b0-4b10-b13d-91c31ea37144) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:23.587619 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Volume 655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41', 'volume_size': 1} Sep 17 21:06:23.588371 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] creating volume 'volume-655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:06:23.589583 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:23.704775 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:06:23.705152 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:06:23.709016 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9159d04b-59b0-4b10-b13d-91c31ea37144) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:23.729993 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9fc481f4-657f-4945-9b7e-03b193fc9938) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:23.753734 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Volume volume-655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41 (655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41): created successfully Sep 17 21:06:23.760145 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9fc481f4-657f-4945-9b7e-03b193fc9938) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:23.780089 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Flow 'volume_create_manager' (91b976ff-7153-4039-b834-d3cbe23e4d19) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:23.803769 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-433ab71f-9326-43cf-947c-5a3762bcb8a1 tempest-VolumesActionsTest-176422482 None] Created volume successfully. Sep 17 21:06:25.098810 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] Acquiring lock "cinder-655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:06:25.109142 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] Lock "cinder-655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:06:25.147163 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:25.202043 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:06:25.202532 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] Volume volume-655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:06:25.222409 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] deleting rbd volume volume-655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:06:25.624176 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] Created reservations ['852b7066-8d4d-414e-b91a-1729693466d5', '422a310d-2ba3-4f95-8f7b-937dabcb4763', '2143f8f6-7e52-4796-a151-adf28d7bcd3f', 'ffb65ec2-b70a-450a-9cb2-851dc9ce2b26'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:06:25.741014 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:25.811718 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:25.884191 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:06:25.890313 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] Deleted volume successfully. Sep 17 21:06:25.914770 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-c5161aef-33c0-4e6d-b761-b956508eb68d tempest-VolumesActionsTest-176422482 None] Lock "cinder-655a0b2d-eba3-47a4-9a9a-dc6bad2d5b41-delete_volume" released by "delete_volume" :: held 0.806s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:06:26.380411 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] Acquiring lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:06:26.397423 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] Lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:06:26.443666 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:26.511989 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:06:26.512635 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] Volume volume-3eb72a32-7071-4518-8571-6d22f73e4404 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:06:26.523149 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] deleting rbd volume volume-3eb72a32-7071-4518-8571-6d22f73e4404 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:06:26.977646 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] Created reservations ['53e7951c-78a7-4c05-ba91-cb12e1307d44', 'a84df9bc-f0f2-4bea-b054-7f73e1a0ffb1', '7dca5d36-e051-408b-9f55-8e3dc1fb36c0', '683bdf98-a3d6-4fb2-bcf4-1de19f160912'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:06:27.041091 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:27.104699 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:27.149968 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:06:27.155157 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] Deleted volume successfully. Sep 17 21:06:27.174867 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-81f635fc-09dd-4edc-97d6-aafcc6d2c445 tempest-VolumesActionsTest-176422482 None] Lock "cinder-3eb72a32-7071-4518-8571-6d22f73e4404-delete_volume" released by "delete_volume" :: held 0.784s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:06:34.257090 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:06:34.278530 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Flow 'volume_create_manager' (a605d838-02b0-4766-8af5-a0d7eb793633) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:34.285765 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0a56d3ab-6571-4561-8210-13526c825970) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:34.305615 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0a56d3ab-6571-4561-8210-13526c825970) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:06:34Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV357Test-Volume-661225286',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f2c10a2d-178f-41ce-9944-335ec2287549,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5816f5fcc813404e93f6fb6c810e38ae',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:06:34Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:06:34Z,use_quota=True,user_id='2e26d30550f34e1aaf0628c3824040a7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:34.312110 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d60804fb-259a-4b13-9c91-f1f97a362842) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:34.318847 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d60804fb-259a-4b13-9c91-f1f97a362842) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:34.326905 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a8b6c6c1-f58e-4d0b-9641-dfe8b2f72439) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:34.335635 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a8b6c6c1-f58e-4d0b-9641-dfe8b2f72439) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f2c10a2d-178f-41ce-9944-335ec2287549', 'volume_name': 'volume-f2c10a2d-178f-41ce-9944-335ec2287549', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:34.341390 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (df37ce88-f9d1-43a4-93d5-323be3a42fd1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:34.346056 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (df37ce88-f9d1-43a4-93d5-323be3a42fd1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:34.351919 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c6a40bdd-37f1-476b-bbbc-12f96ab307f5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:34.353263 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Volume f2c10a2d-178f-41ce-9944-335ec2287549: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f2c10a2d-178f-41ce-9944-335ec2287549', 'volume_size': 1} Sep 17 21:06:34.353933 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] creating volume 'volume-f2c10a2d-178f-41ce-9944-335ec2287549' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:06:34.354935 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:34.467397 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:06:34.468598 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:06:34.473137 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c6a40bdd-37f1-476b-bbbc-12f96ab307f5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f2c10a2d-178f-41ce-9944-335ec2287549', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:34.478642 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6886cb08-06b8-4326-a0c1-4a836d0451c9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:34.493522 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Volume volume-f2c10a2d-178f-41ce-9944-335ec2287549 (f2c10a2d-178f-41ce-9944-335ec2287549): created successfully Sep 17 21:06:34.499556 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6886cb08-06b8-4326-a0c1-4a836d0451c9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:34.507842 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Flow 'volume_create_manager' (a605d838-02b0-4766-8af5-a0d7eb793633) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:34.522425 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f4b3d325-3b3a-4846-818b-47881f68a085 tempest-VolumesTransfersV357Test-80574303 None] Created volume successfully. Sep 17 21:06:36.111791 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e0674f7f-4f41-43b3-a666-5cff5d97d786 tempest-VolumesTransfersV357Test-96005011 None] Transfer volume completed successfully. Sep 17 21:06:36.661203 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] Acquiring lock "cinder-f2c10a2d-178f-41ce-9944-335ec2287549-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:06:36.679897 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] Lock "cinder-f2c10a2d-178f-41ce-9944-335ec2287549-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:06:36.716354 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:36.766611 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:06:36.767036 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] Volume volume-f2c10a2d-178f-41ce-9944-335ec2287549 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:06:36.773256 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] deleting rbd volume volume-f2c10a2d-178f-41ce-9944-335ec2287549 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:06:37.006820 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=58,cinder:UPDATE=28,cinder:DELETE=7,cinder:INSERT=7 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:37.007703 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=56,cinder:UPDATE=39,cinder:INSERT=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:06:37.042695 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] Created reservations ['5c808a71-72fd-4265-990b-9c91ba60ccf6', '0a6c1925-72cf-4488-823e-e59c9244381e', '0f15a3fe-3c32-4190-bcda-837142ec0710', 'cd772916-f7db-4f1f-9b20-cd73cc12b1a0'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:06:37.105192 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:37.132832 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:37.161671 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:06:37.166481 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] Deleted volume successfully. Sep 17 21:06:37.188371 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6afe8a18-ab67-4076-b8ee-3ee961eca3d4 tempest-VolumesTransfersV357Test-21338118 None] Lock "cinder-f2c10a2d-178f-41ce-9944-335ec2287549-delete_volume" released by "delete_volume" :: held 0.509s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:06:38.202171 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:06:38.224812 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Flow 'volume_create_manager' (b08e5857-5d42-4707-a290-e31fe830d451) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:38.232062 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a6d49439-ca53-4949-8db5-275be10c1775) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:38.249993 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a6d49439-ca53-4949-8db5-275be10c1775) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:06:38Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV357Test-Volume-990913086',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1341838b-bbb6-4e24-a24e-5d5aa6f65f69,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5816f5fcc813404e93f6fb6c810e38ae',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:06:38Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:06:38Z,use_quota=True,user_id='2e26d30550f34e1aaf0628c3824040a7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:38.255616 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d66d40a9-2b8a-4d03-8749-ef353620b2d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:38.259767 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d66d40a9-2b8a-4d03-8749-ef353620b2d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:38.268483 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2490db1b-87d7-4887-a254-2174101d99e9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:38.274569 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2490db1b-87d7-4887-a254-2174101d99e9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1341838b-bbb6-4e24-a24e-5d5aa6f65f69', 'volume_name': 'volume-1341838b-bbb6-4e24-a24e-5d5aa6f65f69', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:38.280284 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5289b49c-aca2-4d48-858d-9a83605063ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:38.284693 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5289b49c-aca2-4d48-858d-9a83605063ba) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:38.289555 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (896d9f95-50d8-42fc-a4e1-2448be450790) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:38.290805 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Volume 1341838b-bbb6-4e24-a24e-5d5aa6f65f69: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1341838b-bbb6-4e24-a24e-5d5aa6f65f69', 'volume_size': 1} Sep 17 21:06:38.291368 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] creating volume 'volume-1341838b-bbb6-4e24-a24e-5d5aa6f65f69' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:06:38.292335 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:38.377139 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:06:38.377833 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:06:38.381813 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (896d9f95-50d8-42fc-a4e1-2448be450790) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1341838b-bbb6-4e24-a24e-5d5aa6f65f69', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:38.386197 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8fb44461-ed82-4643-b94a-7cb823e27257) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:38.396418 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Volume volume-1341838b-bbb6-4e24-a24e-5d5aa6f65f69 (1341838b-bbb6-4e24-a24e-5d5aa6f65f69): created successfully Sep 17 21:06:38.403080 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8fb44461-ed82-4643-b94a-7cb823e27257) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:38.408975 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Flow 'volume_create_manager' (b08e5857-5d42-4707-a290-e31fe830d451) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:38.420212 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e2e4ecf3-8886-4a63-b419-9f4fa4770d2d tempest-VolumesTransfersV357Test-80574303 None] Created volume successfully. Sep 17 21:06:39.630738 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] Acquiring lock "cinder-1341838b-bbb6-4e24-a24e-5d5aa6f65f69-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:06:39.659335 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] Lock "cinder-1341838b-bbb6-4e24-a24e-5d5aa6f65f69-delete_volume" acquired by "delete_volume" :: waited 0.031s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:06:39.726818 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:39.786152 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:06:39.786794 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] Volume volume-1341838b-bbb6-4e24-a24e-5d5aa6f65f69 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:06:39.807872 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] deleting rbd volume volume-1341838b-bbb6-4e24-a24e-5d5aa6f65f69 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:06:40.073223 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] Created reservations ['73bcf3f3-8089-4ee8-8315-84abeccfaf88', '4226a720-f594-4529-a055-044e6e6bde33', 'f9ac0712-b895-49a4-aa43-0278437edf1b', '79aed2f1-d900-4a35-9652-d775c9539975'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:06:40.123267 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:40.175141 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:40.212887 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:06:40.218607 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] Deleted volume successfully. Sep 17 21:06:40.234526 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9a5d7c79-020b-471e-b445-0b7dc1e736a2 tempest-VolumesTransfersV357Test-21338118 None] Lock "cinder-1341838b-bbb6-4e24-a24e-5d5aa6f65f69-delete_volume" released by "delete_volume" :: held 0.575s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:06:46.908954 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:06:46.931416 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Flow 'volume_create_manager' (8c6d446c-811e-4e5e-8512-a1a3d129d710) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:46.939053 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bb4c91de-7cc5-493e-9ea1-c51fe59191bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:46.959093 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bb4c91de-7cc5-493e-9ea1-c51fe59191bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:06:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1826427498',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=dd6de8b6-eb47-441a-a79b-306269086b91,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3e857c532a264ea78e24fd8fe1a44fbc',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:06:47Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:06:47Z,use_quota=True,user_id='33784305478f46ae9fdc4d4b82ebc6f5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:46.964451 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (606a5609-f66c-4585-8534-c30774b588b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:46.968518 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (606a5609-f66c-4585-8534-c30774b588b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:46.975563 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dfddbc76-7c06-4325-99ea-f12d49d40d99) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:46.979093 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dfddbc76-7c06-4325-99ea-f12d49d40d99) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'dd6de8b6-eb47-441a-a79b-306269086b91', 'volume_name': 'volume-dd6de8b6-eb47-441a-a79b-306269086b91', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:46.983880 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (57840b1d-6354-4780-90af-3bddb708891d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:46.987894 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (57840b1d-6354-4780-90af-3bddb708891d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:46.992509 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (55b41782-e0ca-4f4f-9d30-000c418141af) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:46.993645 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Volume dd6de8b6-eb47-441a-a79b-306269086b91: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-dd6de8b6-eb47-441a-a79b-306269086b91', 'volume_size': 1} Sep 17 21:06:46.994038 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] creating volume 'volume-dd6de8b6-eb47-441a-a79b-306269086b91' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:06:46.995215 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:47.082788 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:06:47.083193 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:06:47.087449 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (55b41782-e0ca-4f4f-9d30-000c418141af) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-dd6de8b6-eb47-441a-a79b-306269086b91', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:47.092966 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cdc6913e-3e40-4da2-a95b-3c0e8bffa050) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:47.102833 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Volume volume-dd6de8b6-eb47-441a-a79b-306269086b91 (dd6de8b6-eb47-441a-a79b-306269086b91): created successfully Sep 17 21:06:47.107611 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cdc6913e-3e40-4da2-a95b-3c0e8bffa050) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:47.114129 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Flow 'volume_create_manager' (8c6d446c-811e-4e5e-8512-a1a3d129d710) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:47.124546 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3b262a9d-5a7c-4208-bb5c-4720aac4d03c tempest-VolumesActionsTest-905194390 None] Created volume successfully. Sep 17 21:06:50.070928 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:06:50.111330 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Flow 'volume_create_manager' (9ed5f202-3554-47bd-a6b3-b685dbaf1c58) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:50.118882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (75db0c34-8b9d-4072-a9c6-acf5d518fc15) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:50.141012 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (75db0c34-8b9d-4072-a9c6-acf5d518fc15) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:06:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerRescueNegativeTestJSON-volume-429703281',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f337f44a-3587-4bc6-933a-9287690ea825,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f8ed24e4bf6643f7b1474fcec78c4995',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:06:50Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:06:50Z,use_quota=True,user_id='795813bb461e42ad8c2a758e9b8430d6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:50.148978 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9e702244-b47e-497c-9d90-bac4dd9ae4ef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:50.153949 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9e702244-b47e-497c-9d90-bac4dd9ae4ef) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:50.159370 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (df92e9a6-7f18-47ca-b5c6-cc7b70786f2e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:50.163912 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (df92e9a6-7f18-47ca-b5c6-cc7b70786f2e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f337f44a-3587-4bc6-933a-9287690ea825', 'volume_name': 'volume-f337f44a-3587-4bc6-933a-9287690ea825', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:50.169499 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bfac71ef-7132-4115-9568-674032e442fa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:50.186937 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bfac71ef-7132-4115-9568-674032e442fa) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:50.195131 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e753a380-3ea0-4e5b-9aa2-b4a84eae54f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:50.196756 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Volume f337f44a-3587-4bc6-933a-9287690ea825: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f337f44a-3587-4bc6-933a-9287690ea825', 'volume_size': 1} Sep 17 21:06:50.197363 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] creating volume 'volume-f337f44a-3587-4bc6-933a-9287690ea825' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:06:50.198898 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:06:50.327989 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:06:50.328801 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:06:50.333673 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e753a380-3ea0-4e5b-9aa2-b4a84eae54f9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f337f44a-3587-4bc6-933a-9287690ea825', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:50.344117 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (493c7894-999c-4e5c-8f8d-c46467d2886f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:06:50.361198 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Volume volume-f337f44a-3587-4bc6-933a-9287690ea825 (f337f44a-3587-4bc6-933a-9287690ea825): created successfully Sep 17 21:06:50.367775 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (493c7894-999c-4e5c-8f8d-c46467d2886f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:06:50.382142 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Flow 'volume_create_manager' (9ed5f202-3554-47bd-a6b3-b685dbaf1c58) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:06:50.403268 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-97955df3-ae5b-4939-87e2-7a6851b625ef tempest-ServerRescueNegativeTestJSON-533169750 None] Created volume successfully. Sep 17 21:07:01.041134 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:07:01.042148 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:01.068641 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:01.124352 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:07:01.752868 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=23,cinder:SELECT=25,cinder:DELETE=3,cinder:INSERT=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:01.981109 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-31cbbba0-5899-461a-995f-0eb6b3bdcce6 tempest-VolumesActionsTest-905194390 None] Acquiring lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91" by "attach_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:07:01.990364 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-31cbbba0-5899-461a-995f-0eb6b3bdcce6 tempest-VolumesActionsTest-905194390 None] Lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91" acquired by "attach_volume" :: waited 0.010s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:07:02.046014 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-31cbbba0-5899-461a-995f-0eb6b3bdcce6 tempest-VolumesActionsTest-905194390 None] Attaching volume dd6de8b6-eb47-441a-a79b-306269086b91 to instance 95d8ad94-20c4-43ae-be04-087c3ae43d5b at mountpoint /dev/sdb on host None. Sep 17 21:07:02.079394 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-31cbbba0-5899-461a-995f-0eb6b3bdcce6 tempest-VolumesActionsTest-905194390 None] Attach volume completed successfully. Sep 17 21:07:02.084083 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-31cbbba0-5899-461a-995f-0eb6b3bdcce6 tempest-VolumesActionsTest-905194390 None] Lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91" released by "attach_volume" :: held 0.094s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:07:02.226595 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8d42e5b6-aacb-4c1c-8d3c-3fb4ad9ac593 tempest-VolumesActionsTest-905194390 None] Acquiring lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91-detach_volume" by "detach_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:07:02.237639 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8d42e5b6-aacb-4c1c-8d3c-3fb4ad9ac593 tempest-VolumesActionsTest-905194390 None] Lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91-detach_volume" acquired by "detach_volume" :: waited 0.011s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:07:02.238318 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8d42e5b6-aacb-4c1c-8d3c-3fb4ad9ac593 tempest-VolumesActionsTest-905194390 None] Detaching volume dd6de8b6-eb47-441a-a79b-306269086b91 from instance 95d8ad94-20c4-43ae-be04-087c3ae43d5b. Sep 17 21:07:02.292676 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8d42e5b6-aacb-4c1c-8d3c-3fb4ad9ac593 tempest-VolumesActionsTest-905194390 None] Detach volume completed successfully. Sep 17 21:07:02.298417 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8d42e5b6-aacb-4c1c-8d3c-3fb4ad9ac593 tempest-VolumesActionsTest-905194390 None] Lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91-detach_volume" released by "detach_volume" :: held 0.061s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:07:02.660083 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:07:02.686469 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Flow 'volume_create_manager' (060cfe68-b3b9-44fd-b244-11e1a16b2b87) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:07:02.695128 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a6f318ba-a47d-4d8d-84a1-03c8687069d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:02.725861 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a6f318ba-a47d-4d8d-84a1-03c8687069d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:07:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-800505842',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=345bdac6-8146-4c95-b81c-59ac61c7b198,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c3a83c9c29f1483b9ee9c48e94dc17ce',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:07:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:07:03Z,use_quota=True,user_id='24c7a82126124ebb8baf5c656ff9849d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:02.733559 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ddf1abe2-8e50-49bb-be60-1efa4798477d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:02.738480 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ddf1abe2-8e50-49bb-be60-1efa4798477d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:02.743647 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dd145624-6f81-4b9a-ae53-1cdd4572e414) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:02.753264 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dd145624-6f81-4b9a-ae53-1cdd4572e414) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '345bdac6-8146-4c95-b81c-59ac61c7b198', 'volume_name': 'volume-345bdac6-8146-4c95-b81c-59ac61c7b198', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:02.758725 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a0de9e0b-7419-4c83-8e34-699e9c55601c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:02.762798 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a0de9e0b-7419-4c83-8e34-699e9c55601c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:02.767552 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c0106306-0805-4b03-ace7-d5fc0d7ca83f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:02.768720 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Volume 345bdac6-8146-4c95-b81c-59ac61c7b198: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-345bdac6-8146-4c95-b81c-59ac61c7b198', 'volume_size': 1} Sep 17 21:07:02.769228 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] creating volume 'volume-345bdac6-8146-4c95-b81c-59ac61c7b198' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:07:02.769913 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:02.893890 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:07:02.894587 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:07:02.898500 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c0106306-0805-4b03-ace7-d5fc0d7ca83f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-345bdac6-8146-4c95-b81c-59ac61c7b198', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:02.907921 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (691c0889-54fb-473f-bb10-7d134db432c8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:02.921617 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Volume volume-345bdac6-8146-4c95-b81c-59ac61c7b198 (345bdac6-8146-4c95-b81c-59ac61c7b198): created successfully Sep 17 21:07:02.926394 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (691c0889-54fb-473f-bb10-7d134db432c8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:02.933800 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Flow 'volume_create_manager' (060cfe68-b3b9-44fd-b244-11e1a16b2b87) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:07:02.945034 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-45e36de9-aaf5-45d2-9f63-3961c706da7b tempest-VolumesAdminNegativeTest-2142170983 None] Created volume successfully. Sep 17 21:07:04.532254 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-965d4754-61d3-4b5d-b73f-22901a610c28 req-4ce1b62d-9422-49ec-9631-2c351fce5732 tempest-VolumesAdminNegativeTest-2142170983 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:07:05.700366 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-965d4754-61d3-4b5d-b73f-22901a610c28 req-4ce1b62d-9422-49ec-9631-2c351fce5732 tempest-VolumesAdminNegativeTest-2142170983 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.168s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:07:05.702713 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-965d4754-61d3-4b5d-b73f-22901a610c28 req-4ce1b62d-9422-49ec-9631-2c351fce5732 tempest-VolumesAdminNegativeTest-2142170983 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-345bdac6-8146-4c95-b81c-59ac61c7b198', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '345bdac6-8146-4c95-b81c-59ac61c7b198', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:07:05.741164 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-965d4754-61d3-4b5d-b73f-22901a610c28 req-4ce1b62d-9422-49ec-9631-2c351fce5732 tempest-VolumesAdminNegativeTest-2142170983 None] Connection info returned from driver {'name': 'volumes/volume-345bdac6-8146-4c95-b81c-59ac61c7b198', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '345bdac6-8146-4c95-b81c-59ac61c7b198', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'c3d69807-4bae-4912-8b79-0819af9e8f8a'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:07:05.849379 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-965d4754-61d3-4b5d-b73f-22901a610c28 req-4ce1b62d-9422-49ec-9631-2c351fce5732 tempest-VolumesAdminNegativeTest-2142170983 None] attachment_update completed successfully. Sep 17 21:07:09.557219 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=64,cinder:INSERT=6,cinder:UPDATE=26,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:09.574067 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-bd90784e-8e50-4eea-8854-0d80837a94d6 req-c78c752d-c6c7-4ae9-bc52-f8a05b15d5fe tempest-VolumesAdminNegativeTest-2142170983 None] Terminate volume connection completed successfully. Sep 17 21:07:09.574800 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-bd90784e-8e50-4eea-8854-0d80837a94d6 req-c78c752d-c6c7-4ae9-bc52-f8a05b15d5fe tempest-VolumesAdminNegativeTest-2142170983 None] Deleting attachment c3d69807-4bae-4912-8b79-0819af9e8f8a. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:07:13.113660 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:07:13.157707 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Flow 'volume_create_manager' (c4d8a7cb-c050-4f4c-844f-65f557de2a55) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:07:13.164356 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8d2f0102-6660-4eba-8b19-89bf35aa340a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:13.209700 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8d2f0102-6660-4eba-8b19-89bf35aa340a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:07:13Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerRescueNegativeTestJSON-volume-1814741890',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5ba7aae4-11cf-4bae-825a-6ea9f7b052fa,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f8ed24e4bf6643f7b1474fcec78c4995',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:07:13Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:07:13Z,use_quota=True,user_id='795813bb461e42ad8c2a758e9b8430d6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:13.217107 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (626314e4-8727-4241-9fd6-75286aa669e3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:13.222736 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (626314e4-8727-4241-9fd6-75286aa669e3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:13.228712 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (530826df-2ddf-4664-af0f-685e71141dd4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:13.233063 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (530826df-2ddf-4664-af0f-685e71141dd4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5ba7aae4-11cf-4bae-825a-6ea9f7b052fa', 'volume_name': 'volume-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:13.238522 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (27307bd4-33dc-4e4c-b3c0-dbf64e66a96b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:13.257968 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (27307bd4-33dc-4e4c-b3c0-dbf64e66a96b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:13.262653 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (edb714ab-4b5c-428b-ace9-1f62853f2afc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:13.263782 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Volume 5ba7aae4-11cf-4bae-825a-6ea9f7b052fa: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa', 'volume_size': 1} Sep 17 21:07:13.264946 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] creating volume 'volume-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:07:13.264946 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:13.412589 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:07:13.413321 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:07:13.421804 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (edb714ab-4b5c-428b-ace9-1f62853f2afc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:13.427348 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4bab7d53-b83b-45f1-a5ad-75f04cc29dbd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:13.443076 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Volume volume-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa (5ba7aae4-11cf-4bae-825a-6ea9f7b052fa): created successfully Sep 17 21:07:13.451250 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4bab7d53-b83b-45f1-a5ad-75f04cc29dbd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:13.458783 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Flow 'volume_create_manager' (c4d8a7cb-c050-4f4c-844f-65f557de2a55) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:07:13.472388 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-1107876d-1194-48eb-916d-9c4bb29ea3bd tempest-ServerRescueNegativeTestJSON-533169750 None] Created volume successfully. Sep 17 21:07:17.144478 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:07:17.170963 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Flow 'volume_create_manager' (f64283e9-2d2a-462e-a4e1-b9b911c080ef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:07:17.178057 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ec2a28b5-dece-4c9b-a01a-8b3848fae27e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:17.197389 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ec2a28b5-dece-4c9b-a01a-8b3848fae27e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:07:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-319360219',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6fa7e14863254ab3964ac907c168d9e6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:07:17Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:07:17Z,use_quota=True,user_id='c544e8e6f7a741c985c92ed7437397cd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:17.203668 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (af62b426-bce3-45d0-bbc0-36aae02afa84) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:17.228523 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (af62b426-bce3-45d0-bbc0-36aae02afa84) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:17.233608 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (db49b070-42a5-4297-81d8-d962933d34db) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:17.242312 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (db49b070-42a5-4297-81d8-d962933d34db) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd1c88125-bbdc-4e6f-99b5-cc2bbb27caf2', 'volume_name': 'volume-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:17.247594 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (49f73f1b-28b7-4d6f-b35c-0b8bcd92f0f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:17.251689 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (49f73f1b-28b7-4d6f-b35c-0b8bcd92f0f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:17.268191 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (62d95ff6-8577-480e-9c15-6d3f753bf2a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:17.271786 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Volume d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2', 'volume_size': 1} Sep 17 21:07:17.272187 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] creating volume 'volume-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:07:17.272850 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:17.385258 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:07:17.385974 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:07:17.390226 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (62d95ff6-8577-480e-9c15-6d3f753bf2a8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:17.396808 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8f40044c-82da-4bf8-b3f7-5b4d9d19d64d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:17.415328 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Volume volume-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2 (d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2): created successfully Sep 17 21:07:17.421572 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8f40044c-82da-4bf8-b3f7-5b4d9d19d64d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:17.428756 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Flow 'volume_create_manager' (f64283e9-2d2a-462e-a4e1-b9b911c080ef) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:07:17.446881 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a90f9926-0b17-4da2-a461-93486709f7e8 tempest-AttachVolumeNegativeTest-1020448628 None] Created volume successfully. Sep 17 21:07:18.918600 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-d1b5b23c-2e05-493b-a964-923d9ca148d4 req-2d9fb1ce-b9fb-4374-98c5-c7f63f32e36d tempest-AttachVolumeNegativeTest-1020448628 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:07:19.999152 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-d1b5b23c-2e05-493b-a964-923d9ca148d4 req-2d9fb1ce-b9fb-4374-98c5-c7f63f32e36d tempest-AttachVolumeNegativeTest-1020448628 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.080s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:07:20.000163 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-d1b5b23c-2e05-493b-a964-923d9ca148d4 req-2d9fb1ce-b9fb-4374-98c5-c7f63f32e36d tempest-AttachVolumeNegativeTest-1020448628 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'd1c88125-bbdc-4e6f-99b5-cc2bbb27caf2', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:07:20.059541 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-d1b5b23c-2e05-493b-a964-923d9ca148d4 req-2d9fb1ce-b9fb-4374-98c5-c7f63f32e36d tempest-AttachVolumeNegativeTest-1020448628 None] Connection info returned from driver {'name': 'volumes/volume-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'd1c88125-bbdc-4e6f-99b5-cc2bbb27caf2', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '9bb65986-dd41-4ec7-89b7-83b3a0f09844'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:07:20.124878 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-d1b5b23c-2e05-493b-a964-923d9ca148d4 req-2d9fb1ce-b9fb-4374-98c5-c7f63f32e36d tempest-AttachVolumeNegativeTest-1020448628 None] attachment_update completed successfully. Sep 17 21:07:25.239807 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:07:25.269257 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Flow 'volume_create_manager' (dec4b4be-9632-49c6-ab6a-6b65a19ebdc0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:07:25.278776 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5ded7e61-543c-4196-9ad1-5d8fec9b45ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:25.301242 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5ded7e61-543c-4196-9ad1-5d8fec9b45ba) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:07:25Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-202249657',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ea58986c-9d33-41ac-ae50-266d4bab26b7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c3a83c9c29f1483b9ee9c48e94dc17ce',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:07:25Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:07:25Z,use_quota=True,user_id='24c7a82126124ebb8baf5c656ff9849d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:25.306682 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (77873380-5469-440d-b279-14ab53601651) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:25.311026 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (77873380-5469-440d-b279-14ab53601651) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:25.316856 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (90e0faf4-79e9-4e91-a371-a96e2d614a5b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:25.321167 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (90e0faf4-79e9-4e91-a371-a96e2d614a5b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ea58986c-9d33-41ac-ae50-266d4bab26b7', 'volume_name': 'volume-ea58986c-9d33-41ac-ae50-266d4bab26b7', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:25.340078 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1f919926-89bd-4ddb-9909-8a35492514cb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:25.345202 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1f919926-89bd-4ddb-9909-8a35492514cb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:25.350081 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (856ca43a-b9d5-4881-adde-84ca1bc0f5e6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:25.351486 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Volume ea58986c-9d33-41ac-ae50-266d4bab26b7: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ea58986c-9d33-41ac-ae50-266d4bab26b7', 'volume_size': 1} Sep 17 21:07:25.352095 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] creating volume 'volume-ea58986c-9d33-41ac-ae50-266d4bab26b7' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:07:25.353103 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:25.452526 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:07:25.453003 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:07:25.456815 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (856ca43a-b9d5-4881-adde-84ca1bc0f5e6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ea58986c-9d33-41ac-ae50-266d4bab26b7', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:25.461690 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (32e9b913-387c-4787-b529-70b9221b69d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:07:25.478710 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Volume volume-ea58986c-9d33-41ac-ae50-266d4bab26b7 (ea58986c-9d33-41ac-ae50-266d4bab26b7): created successfully Sep 17 21:07:25.483133 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (32e9b913-387c-4787-b529-70b9221b69d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:07:25.491513 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Flow 'volume_create_manager' (dec4b4be-9632-49c6-ab6a-6b65a19ebdc0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:07:25.510645 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a7a674da-5064-4ee2-931f-dd41801b113e tempest-VolumesAdminNegativeTest-2142170983 None] Created volume successfully. Sep 17 21:07:27.749389 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] Acquiring lock "cinder-ea58986c-9d33-41ac-ae50-266d4bab26b7-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:07:27.776557 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] Lock "cinder-ea58986c-9d33-41ac-ae50-266d4bab26b7-delete_volume" acquired by "delete_volume" :: waited 0.027s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:07:27.869363 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:27.979352 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:07:27.980032 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] Volume volume-ea58986c-9d33-41ac-ae50-266d4bab26b7 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:07:27.995797 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] deleting rbd volume volume-ea58986c-9d33-41ac-ae50-266d4bab26b7 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:07:28.334520 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] Created reservations ['a4164cb7-3789-4e65-bdda-a94d5a7ab5d8', '3d6474d5-a853-4d47-8e45-271aa398d5b1', '1f07cb93-0fe6-4b0a-bc64-ceea18015dba', '14facd90-5d2b-402a-8d60-afa932e7bf77'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:07:28.398525 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:28.438906 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:28.540004 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:07:28.546186 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] Deleted volume successfully. Sep 17 21:07:28.588277 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-adf5b18a-2496-4ea3-9585-c73d3fd2b674 tempest-VolumesAdminNegativeTest-2142170983 None] Lock "cinder-ea58986c-9d33-41ac-ae50-266d4bab26b7-delete_volume" released by "delete_volume" :: held 0.811s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:07:29.468968 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d8c048dd-449a-4e8d-b89a-1d9beba2bbac tempest-VolumesActionsTest-905194390 None] Acquiring lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91" by "attach_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:07:29.483757 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d8c048dd-449a-4e8d-b89a-1d9beba2bbac tempest-VolumesActionsTest-905194390 None] Lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91" acquired by "attach_volume" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:07:29.526742 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d8c048dd-449a-4e8d-b89a-1d9beba2bbac tempest-VolumesActionsTest-905194390 None] Attaching volume dd6de8b6-eb47-441a-a79b-306269086b91 to instance 964a7041-9baa-42fb-a51c-f760437411df at mountpoint /dev/sdb on host None. Sep 17 21:07:29.559450 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d8c048dd-449a-4e8d-b89a-1d9beba2bbac tempest-VolumesActionsTest-905194390 None] Attach volume completed successfully. Sep 17 21:07:29.570859 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d8c048dd-449a-4e8d-b89a-1d9beba2bbac tempest-VolumesActionsTest-905194390 None] Lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91" released by "attach_volume" :: held 0.087s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:07:29.827032 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-43e2c50f-f956-45ce-aadc-829236b4f3b0 tempest-VolumesActionsTest-905194390 None] Acquiring lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91-detach_volume" by "detach_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:07:29.837491 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-43e2c50f-f956-45ce-aadc-829236b4f3b0 tempest-VolumesActionsTest-905194390 None] Lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91-detach_volume" acquired by "detach_volume" :: waited 0.010s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:07:29.838315 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-43e2c50f-f956-45ce-aadc-829236b4f3b0 tempest-VolumesActionsTest-905194390 None] Detaching volume dd6de8b6-eb47-441a-a79b-306269086b91 from instance 964a7041-9baa-42fb-a51c-f760437411df. Sep 17 21:07:29.864984 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=74,cinder:INSERT=2,cinder:UPDATE=24 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:29.887907 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-43e2c50f-f956-45ce-aadc-829236b4f3b0 tempest-VolumesActionsTest-905194390 None] Detach volume completed successfully. Sep 17 21:07:29.892779 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-43e2c50f-f956-45ce-aadc-829236b4f3b0 tempest-VolumesActionsTest-905194390 None] Lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91-detach_volume" released by "detach_volume" :: held 0.056s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:07:36.585901 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] Acquiring lock "cinder-345bdac6-8146-4c95-b81c-59ac61c7b198-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:07:36.598184 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] Lock "cinder-345bdac6-8146-4c95-b81c-59ac61c7b198-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:07:36.643464 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:36.735089 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:07:36.735513 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] Volume volume-345bdac6-8146-4c95-b81c-59ac61c7b198 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:07:36.745230 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] deleting rbd volume volume-345bdac6-8146-4c95-b81c-59ac61c7b198 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:07:36.883609 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-adbafad8-d046-4a15-a069-0872fcb2af89 tempest-VolumesActionsTest-905194390 None] Running cmd (subprocess): rbd export --pool volumes volume-dd6de8b6-eb47-441a-a79b-306269086b91 /opt/stack/data/cinder/conversion/volume-dd6de8b6-eb47-441a-a79b-306269086b91-76ac6df9-e5ab-4740-93fa-0c543130679e --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:07:37.120123 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] Created reservations ['a54eeddf-b59d-4770-b201-7924d90f6bad', '7fdba5ff-e76a-4363-9f7c-ea9a68aeb185', 'ed60ab51-6699-4a45-9c97-c0af4f237ba0', 'ee573fe0-0650-49c9-9f53-fd8d16f4372b'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:07:37.172246 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=66,cinder:UPDATE=23,cinder:DELETE=4,cinder:INSERT=7 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:07:37.209662 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:37.270577 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:07:37.328937 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:07:37.337416 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] Deleted volume successfully. Sep 17 21:07:37.374933 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-86927868-59e2-4ba3-893d-45b7f603547c tempest-VolumesAdminNegativeTest-2142170983 None] Lock "cinder-345bdac6-8146-4c95-b81c-59ac61c7b198-delete_volume" released by "delete_volume" :: held 0.777s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:07:38.430515 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-adbafad8-d046-4a15-a069-0872fcb2af89 tempest-VolumesActionsTest-905194390 None] CMD "rbd export --pool volumes volume-dd6de8b6-eb47-441a-a79b-306269086b91 /opt/stack/data/cinder/conversion/volume-dd6de8b6-eb47-441a-a79b-306269086b91-76ac6df9-e5ab-4740-93fa-0c543130679e --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.549s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:07:38.444966 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.image_utils [None req-adbafad8-d046-4a15-a069-0872fcb2af89 tempest-VolumesActionsTest-905194390 None] 76ac6df9-e5ab-4740-93fa-0c543130679e was raw, no need to convert to raw {{(pid=133409) upload_volume /opt/stack/cinder/cinder/image/image_utils.py:851}} Sep 17 21:08:01.040660 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:08:01.041343 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:08:01.135959 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:08:01.228343 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:08:01.763089 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=21,cinder:UPDATE=12,cinder:INSERT=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:05.985726 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:08:06.020370 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Flow 'volume_create_manager' (19eda7dc-8858-44c0-8e53-a7df62a68e0a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:08:06.027731 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (37c4984c-12b0-4f32-955f-6f33fde35e20) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:06.059495 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (37c4984c-12b0-4f32-955f-6f33fde35e20) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:08:05Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-513107934',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c9e0dd22-ca21-4935-99e2-ec55ffcac017,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c4a4d7fd12784db69cf699e7e58d4d6a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:08:06Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:08:06Z,use_quota=True,user_id='7a13dc0c95a64d499bd05f808565fccd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:06.065203 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5f9ed958-f634-44ed-b9fd-0484e6292658) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:06.071514 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5f9ed958-f634-44ed-b9fd-0484e6292658) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:06.076839 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (03d20ad6-3f4e-4337-b73b-3252bdab1917) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:06.260795 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (03d20ad6-3f4e-4337-b73b-3252bdab1917) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'c9e0dd22-ca21-4935-99e2-ec55ffcac017', 'volume_name': 'volume-c9e0dd22-ca21-4935-99e2-ec55ffcac017', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:06.283405 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6a4aef88-ef2c-413f-9fd0-4c2fee8561e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:06.287840 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6a4aef88-ef2c-413f-9fd0-4c2fee8561e1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:06.300579 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (25c4b246-a3bf-41ae-9e11-13800f79bf55) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:06.301776 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Volume c9e0dd22-ca21-4935-99e2-ec55ffcac017: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-c9e0dd22-ca21-4935-99e2-ec55ffcac017', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 21:08:06.302365 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Cloning c9e0dd22-ca21-4935-99e2-ec55ffcac017 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 21:08:06.303150 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:08:06.319781 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:08:06.338454 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 8, 6, 325939)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:08:06.339169 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:08:06.339893 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 21:08:06.346797 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.028s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:08:06.348430 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 21:08:06.359402 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 8, 6, 354480)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:08:06.360028 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:08:06.360525 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 21:08:06.376461 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:08:06.487140 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] creating snapshot='volume-c9e0dd22-ca21-4935-99e2-ec55ffcac017.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:08:06.752550 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-c9e0dd22-ca21-4935-99e2-ec55ffcac017.clone_snap' to 'volume-c9e0dd22-ca21-4935-99e2-ec55ffcac017' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:08:07.117798 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:08:07.119091 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:08:07.119457 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:08:07.151260 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:08:07.151894 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Marking volume c9e0dd22-ca21-4935-99e2-ec55ffcac017 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:08:07.165481 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to c9e0dd22-ca21-4935-99e2-ec55ffcac017. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 21:08:07.166212 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Creating volume glance metadata for volume c9e0dd22-ca21-4935-99e2-ec55ffcac017 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 21:08:07.274530 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (25c4b246-a3bf-41ae-9e11-13800f79bf55) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c9e0dd22-ca21-4935-99e2-ec55ffcac017', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:07.286944 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ee5e7e1a-7f70-470b-b775-4bd0d30c0c5e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:07.301223 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:DELETE=1,cinder:SELECT=15,cinder:UPDATE=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:07.315909 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Volume volume-c9e0dd22-ca21-4935-99e2-ec55ffcac017 (c9e0dd22-ca21-4935-99e2-ec55ffcac017): created successfully Sep 17 21:08:07.319495 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ee5e7e1a-7f70-470b-b775-4bd0d30c0c5e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:07.334357 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Flow 'volume_create_manager' (19eda7dc-8858-44c0-8e53-a7df62a68e0a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:08:07.360846 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-db1ac843-0edc-49c2-9eac-2099783c6b0f tempest-TaggedBootDevicesTest_v242-970179712 None] Created volume successfully. Sep 17 21:08:09.078388 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:08:09.111477 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Flow 'volume_create_manager' (44df3f51-2bbc-4927-8e80-ef518e72d022) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:08:09.124093 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c7d7bc14-75b4-4beb-8e34-530d70a8cc17) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:09.207055 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c7d7bc14-75b4-4beb-8e34-530d70a8cc17) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:08:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-830204972',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=03debbe2-b045-44c5-a1ec-8136363e731b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c4a4d7fd12784db69cf699e7e58d4d6a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:08:09Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:08:09Z,use_quota=True,user_id='7a13dc0c95a64d499bd05f808565fccd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:09.229335 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (aeea71f9-e691-4901-af8d-36e66bcde5ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:09.279261 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (aeea71f9-e691-4901-af8d-36e66bcde5ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:09.295351 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (59456185-2ff7-4891-a4ed-6f6b76c290cf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:09.304277 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (59456185-2ff7-4891-a4ed-6f6b76c290cf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '03debbe2-b045-44c5-a1ec-8136363e731b', 'volume_name': 'volume-03debbe2-b045-44c5-a1ec-8136363e731b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:09.325171 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (10dfd889-e6a6-4ac7-a523-5ad3984e99d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:09.336513 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (10dfd889-e6a6-4ac7-a523-5ad3984e99d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:09.355541 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (42ebcb79-adb5-4248-a0cd-0ec6b238b39b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:09.370167 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Volume 03debbe2-b045-44c5-a1ec-8136363e731b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-03debbe2-b045-44c5-a1ec-8136363e731b', 'volume_size': 1} Sep 17 21:08:09.376302 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] creating volume 'volume-03debbe2-b045-44c5-a1ec-8136363e731b' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:08:09.382967 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:08:09.626738 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:08:09.626738 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:08:09.642293 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (42ebcb79-adb5-4248-a0cd-0ec6b238b39b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-03debbe2-b045-44c5-a1ec-8136363e731b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:09.663051 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (94e0be59-2f94-44da-a5ad-d19985f2dafe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:09.692518 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Volume volume-03debbe2-b045-44c5-a1ec-8136363e731b (03debbe2-b045-44c5-a1ec-8136363e731b): created successfully Sep 17 21:08:09.737561 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (94e0be59-2f94-44da-a5ad-d19985f2dafe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:09.759865 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Flow 'volume_create_manager' (44df3f51-2bbc-4927-8e80-ef518e72d022) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:08:09.790224 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e611eae2-673c-495b-a167-a705c9155cca tempest-TaggedBootDevicesTest_v242-970179712 None] Created volume successfully. Sep 17 21:08:10.844465 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:08:10.925073 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Flow 'volume_create_manager' (48b26010-8a03-4f0d-8ec7-4ceb27ca23d5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:08:10.943099 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a6309a51-49ff-4465-be7a-554d31373285) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:10.991998 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a6309a51-49ff-4465-be7a-554d31373285) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:08:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-63652676',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=63216c51-2591-4185-b165-a1abff51874d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c4a4d7fd12784db69cf699e7e58d4d6a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:08:11Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:08:11Z,use_quota=True,user_id='7a13dc0c95a64d499bd05f808565fccd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:11.009179 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0d8fed05-5d84-411f-9897-62e9a8ffde1e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:11.021500 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0d8fed05-5d84-411f-9897-62e9a8ffde1e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:11.034421 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3c59066b-ea29-4374-8673-354bebf3b9c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:11.045828 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3c59066b-ea29-4374-8673-354bebf3b9c5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '63216c51-2591-4185-b165-a1abff51874d', 'volume_name': 'volume-63216c51-2591-4185-b165-a1abff51874d', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:11.050955 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1fdf6f99-d476-426b-b166-5f76b9a704e9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:11.069683 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1fdf6f99-d476-426b-b166-5f76b9a704e9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:11.080378 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6aa6dfdb-3587-4903-aba0-0671aa84d021) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:11.085179 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Volume 63216c51-2591-4185-b165-a1abff51874d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-63216c51-2591-4185-b165-a1abff51874d', 'volume_size': 1} Sep 17 21:08:11.086415 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] creating volume 'volume-63216c51-2591-4185-b165-a1abff51874d' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:08:11.088563 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:08:11.272585 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:08:11.274267 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:08:11.287620 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6aa6dfdb-3587-4903-aba0-0671aa84d021) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-63216c51-2591-4185-b165-a1abff51874d', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:11.298596 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bca71fb0-b767-4fcd-b62a-676f991aabdb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:11.332305 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Volume volume-63216c51-2591-4185-b165-a1abff51874d (63216c51-2591-4185-b165-a1abff51874d): created successfully Sep 17 21:08:11.339294 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bca71fb0-b767-4fcd-b62a-676f991aabdb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:11.347079 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Flow 'volume_create_manager' (48b26010-8a03-4f0d-8ec7-4ceb27ca23d5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:08:11.379989 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6d30ef50-0c07-42d5-8c8c-e5c386f49a95 tempest-TaggedBootDevicesTest_v242-970179712 None] Created volume successfully. Sep 17 21:08:22.725401 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-adbafad8-d046-4a15-a069-0872fcb2af89 tempest-VolumesActionsTest-905194390 None] Uploaded volume to glance image-id: 76ac6df9-e5ab-4740-93fa-0c543130679e. {{(pid=133409) copy_volume_to_image /opt/stack/cinder/cinder/volume/manager.py:1791}} Sep 17 21:08:22.753720 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-adbafad8-d046-4a15-a069-0872fcb2af89 tempest-VolumesActionsTest-905194390 None] Copy volume to image completed successfully. Sep 17 21:08:24.425406 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-905303f8-0f3a-47d5-b62d-5eee02443d13 req-86ad6fb2-729f-476b-9735-c14ba029b105 tempest-ServerRescueNegativeTestJSON-533169750 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:08:24.523313 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] Acquiring lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:08:24.561696 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] Lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91-delete_volume" acquired by "delete_volume" :: waited 0.038s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:08:24.684135 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:08:24.829492 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:08:24.829967 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] Volume volume-dd6de8b6-eb47-441a-a79b-306269086b91 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:08:24.844162 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] deleting rbd volume volume-dd6de8b6-eb47-441a-a79b-306269086b91 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:08:25.251675 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] Created reservations ['11f563d0-2102-4151-9db8-f71ea1b2b191', '614a941f-8064-454b-9f99-f8fcfd8199e1', '9a002cc4-578d-4dc0-8fd6-44842e9b01ee', '9f824a18-e241-4fe8-b5f7-eeddb982af58'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:08:25.391624 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:08:25.486310 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:08:25.568736 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:08:25.578745 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] Deleted volume successfully. Sep 17 21:08:25.606723 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5a694978-5181-4729-bb46-bb33b92ee065 tempest-VolumesActionsTest-905194390 None] Lock "cinder-dd6de8b6-eb47-441a-a79b-306269086b91-delete_volume" released by "delete_volume" :: held 1.046s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:08:26.356603 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-905303f8-0f3a-47d5-b62d-5eee02443d13 req-86ad6fb2-729f-476b-9735-c14ba029b105 tempest-ServerRescueNegativeTestJSON-533169750 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.934s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:08:26.357394 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-905303f8-0f3a-47d5-b62d-5eee02443d13 req-86ad6fb2-729f-476b-9735-c14ba029b105 tempest-ServerRescueNegativeTestJSON-533169750 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '5ba7aae4-11cf-4bae-825a-6ea9f7b052fa', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:08:26.388178 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-905303f8-0f3a-47d5-b62d-5eee02443d13 req-86ad6fb2-729f-476b-9735-c14ba029b105 tempest-ServerRescueNegativeTestJSON-533169750 None] Connection info returned from driver {'name': 'volumes/volume-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '5ba7aae4-11cf-4bae-825a-6ea9f7b052fa', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'a630e7ce-6cdd-4e32-969b-b753303919ab'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:08:26.486244 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-905303f8-0f3a-47d5-b62d-5eee02443d13 req-86ad6fb2-729f-476b-9735-c14ba029b105 tempest-ServerRescueNegativeTestJSON-533169750 None] attachment_update completed successfully. Sep 17 21:08:31.690822 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:08:31.721407 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Flow 'volume_create_manager' (90949cd2-cb51-4ca2-86bc-d1a0388363c2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:08:31.728604 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (310c5d1e-f13a-4277-9ae5-d2eecbadfb13) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:31.749490 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (310c5d1e-f13a-4277-9ae5-d2eecbadfb13) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:08:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1935560706',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ab2daf87-1564-4764-b491-1e5192ed5de8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='42e1e6b096bb41419c594dcde97e16e7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:08:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:08:32Z,use_quota=True,user_id='6d03e73a3bed48f6bf61a702f0f41658',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:31.755466 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2b0ca533-f45c-4d92-9377-2887630367bc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:31.760435 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2b0ca533-f45c-4d92-9377-2887630367bc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:31.765967 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e639f5a3-d3dc-42ab-9dc7-1e7980328fe2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:31.770367 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e639f5a3-d3dc-42ab-9dc7-1e7980328fe2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ab2daf87-1564-4764-b491-1e5192ed5de8', 'volume_name': 'volume-ab2daf87-1564-4764-b491-1e5192ed5de8', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:31.776448 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b32ffb99-d6e4-402f-85ff-99ad940c0008) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:31.780729 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b32ffb99-d6e4-402f-85ff-99ad940c0008) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:31.785856 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (68b8138e-88da-44b0-9108-6a6a7c29e2f8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:31.787174 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Volume ab2daf87-1564-4764-b491-1e5192ed5de8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ab2daf87-1564-4764-b491-1e5192ed5de8', 'volume_size': 1} Sep 17 21:08:31.787556 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] creating volume 'volume-ab2daf87-1564-4764-b491-1e5192ed5de8' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:08:31.788505 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:08:31.879701 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=52,cinder:UPDATE=13,cinder:INSERT=12,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:31.933947 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:08:31.934373 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:08:31.940506 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (68b8138e-88da-44b0-9108-6a6a7c29e2f8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ab2daf87-1564-4764-b491-1e5192ed5de8', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:31.955951 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ac28626c-736e-48ad-9404-6b618be4fe83) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:08:31.972490 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Volume volume-ab2daf87-1564-4764-b491-1e5192ed5de8 (ab2daf87-1564-4764-b491-1e5192ed5de8): created successfully Sep 17 21:08:31.976554 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ac28626c-736e-48ad-9404-6b618be4fe83) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:08:31.981873 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Flow 'volume_create_manager' (90949cd2-cb51-4ca2-86bc-d1a0388363c2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:08:32.003437 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-65887228-9159-4e8d-8929-7fba99b92fdf tempest-VolumesBackupsTest-615004682 None] Created volume successfully. Sep 17 21:08:40.956841 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=50,cinder:DELETE=3,cinder:UPDATE=23,cinder:INSERT=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:08:40.967112 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-c1ec67ae-061c-4fdc-9be5-b3cc1cb0d1af req-ba086b43-baed-460e-a321-d50ccf4e995f tempest-AttachVolumeNegativeTest-1020448628 None] Terminate volume connection completed successfully. Sep 17 21:08:40.967308 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-c1ec67ae-061c-4fdc-9be5-b3cc1cb0d1af req-ba086b43-baed-460e-a321-d50ccf4e995f tempest-AttachVolumeNegativeTest-1020448628 None] Deleting attachment 9bb65986-dd41-4ec7-89b7-83b3a0f09844. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:08:43.775358 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-8015916b-29f0-4d2c-8f19-4fde71cf6855 tempest-TaggedBootDevicesTest_v242-970179712 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:08:45.035089 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-8015916b-29f0-4d2c-8f19-4fde71cf6855 tempest-TaggedBootDevicesTest_v242-970179712 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.259s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:08:45.038915 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-8015916b-29f0-4d2c-8f19-4fde71cf6855 tempest-TaggedBootDevicesTest_v242-970179712 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-c9e0dd22-ca21-4935-99e2-ec55ffcac017', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'c9e0dd22-ca21-4935-99e2-ec55ffcac017', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:08:45.112906 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-8015916b-29f0-4d2c-8f19-4fde71cf6855 tempest-TaggedBootDevicesTest_v242-970179712 None] Connection info returned from driver {'name': 'volumes/volume-c9e0dd22-ca21-4935-99e2-ec55ffcac017', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'c9e0dd22-ca21-4935-99e2-ec55ffcac017', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '1c7b252e-c0f1-4851-a2fb-a018a2d2b497'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:08:45.252942 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-8015916b-29f0-4d2c-8f19-4fde71cf6855 tempest-TaggedBootDevicesTest_v242-970179712 None] attachment_update completed successfully. Sep 17 21:08:45.773684 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-1aed4bf7-97b7-41b1-aad7-a85cf465e952 tempest-TaggedBootDevicesTest_v242-970179712 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:08:46.912020 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-1aed4bf7-97b7-41b1-aad7-a85cf465e952 tempest-TaggedBootDevicesTest_v242-970179712 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.138s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:08:46.913678 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-1aed4bf7-97b7-41b1-aad7-a85cf465e952 tempest-TaggedBootDevicesTest_v242-970179712 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-03debbe2-b045-44c5-a1ec-8136363e731b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '03debbe2-b045-44c5-a1ec-8136363e731b', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:08:46.960532 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-1aed4bf7-97b7-41b1-aad7-a85cf465e952 tempest-TaggedBootDevicesTest_v242-970179712 None] Connection info returned from driver {'name': 'volumes/volume-03debbe2-b045-44c5-a1ec-8136363e731b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '03debbe2-b045-44c5-a1ec-8136363e731b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '0e4663f0-c4da-45d4-abd5-12f2defcec88'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:08:47.099093 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-1aed4bf7-97b7-41b1-aad7-a85cf465e952 tempest-TaggedBootDevicesTest_v242-970179712 None] attachment_update completed successfully. Sep 17 21:08:47.537086 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-6ee6b501-4d48-4ec1-b7cd-2a3bebc81ca5 tempest-TaggedBootDevicesTest_v242-970179712 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:08:48.693475 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-6ee6b501-4d48-4ec1-b7cd-2a3bebc81ca5 tempest-TaggedBootDevicesTest_v242-970179712 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.156s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:08:48.694697 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-6ee6b501-4d48-4ec1-b7cd-2a3bebc81ca5 tempest-TaggedBootDevicesTest_v242-970179712 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-63216c51-2591-4185-b165-a1abff51874d', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '63216c51-2591-4185-b165-a1abff51874d', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:08:48.755497 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-6ee6b501-4d48-4ec1-b7cd-2a3bebc81ca5 tempest-TaggedBootDevicesTest_v242-970179712 None] Connection info returned from driver {'name': 'volumes/volume-63216c51-2591-4185-b165-a1abff51874d', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '63216c51-2591-4185-b165-a1abff51874d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '693624fa-7fb5-4bdd-ab09-a5049c496985'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:08:48.875570 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-3e3ffa5c-65f0-44a3-ad42-c6a8e1f20fe7 req-6ee6b501-4d48-4ec1-b7cd-2a3bebc81ca5 tempest-TaggedBootDevicesTest_v242-970179712 None] attachment_update completed successfully. Sep 17 21:09:01.037217 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:09:01.038456 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:09:01.102813 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:09:01.189310 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:09:01.895563 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=34,cinder:UPDATE=6 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:03.682743 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=43,cinder:DELETE=1,cinder:UPDATE=6 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:03.695912 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-4297a356-3fde-4ce0-888d-4eda8959bf07 req-c0f0f17c-a4eb-46a7-9e47-3cddac443131 tempest-VolumesBackupsTest-615004682 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:09:04.711990 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-4297a356-3fde-4ce0-888d-4eda8959bf07 req-c0f0f17c-a4eb-46a7-9e47-3cddac443131 tempest-VolumesBackupsTest-615004682 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.016s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:09:04.712913 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-4297a356-3fde-4ce0-888d-4eda8959bf07 req-c0f0f17c-a4eb-46a7-9e47-3cddac443131 tempest-VolumesBackupsTest-615004682 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-ab2daf87-1564-4764-b491-1e5192ed5de8', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'ab2daf87-1564-4764-b491-1e5192ed5de8', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:09:04.751293 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-4297a356-3fde-4ce0-888d-4eda8959bf07 req-c0f0f17c-a4eb-46a7-9e47-3cddac443131 tempest-VolumesBackupsTest-615004682 None] Connection info returned from driver {'name': 'volumes/volume-ab2daf87-1564-4764-b491-1e5192ed5de8', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'ab2daf87-1564-4764-b491-1e5192ed5de8', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '3fe0c97e-31e7-4a8e-94d8-383b15c040eb'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:09:04.825848 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-4297a356-3fde-4ce0-888d-4eda8959bf07 req-c0f0f17c-a4eb-46a7-9e47-3cddac443131 tempest-VolumesBackupsTest-615004682 None] attachment_update completed successfully. Sep 17 21:09:07.561113 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f4ecb8fc-be6f-4f03-a1c6-6c369b022462 tempest-VolumesBackupsTest-615004682 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-17T21:09:07Z,data_timestamp=2022-09-17T21:09:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-981470936',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=1bcf2150-2009-42cc-ae2f-24775c7a377a,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='42e1e6b096bb41419c594dcde97e16e7',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='6d03e73a3bed48f6bf61a702f0f41658',volume_id=ab2daf87-1564-4764-b491-1e5192ed5de8) Sep 17 21:09:07.581141 ncdv-master-1 cinder-volume[133409]: INFO cinder.rpc [None req-f4ecb8fc-be6f-4f03-a1c6-6c369b022462 tempest-VolumesBackupsTest-615004682 None] Automatically selected cinder-backup objects version 1.39 as minimum service version. Sep 17 21:09:07.592574 ncdv-master-1 cinder-volume[133409]: INFO cinder.rpc [None req-f4ecb8fc-be6f-4f03-a1c6-6c369b022462 tempest-VolumesBackupsTest-615004682 None] Automatically selected cinder-backup RPC version 2.3 as minimum service version. Sep 17 21:09:07.593959 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.backup.rpcapi [None req-f4ecb8fc-be6f-4f03-a1c6-6c369b022462 tempest-VolumesBackupsTest-615004682 None] continue_backup in rpcapi backup_id 1bcf2150-2009-42cc-ae2f-24775c7a377a {{(pid=133409) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 17 21:09:18.025269 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-f4ecb8fc-be6f-4f03-a1c6-6c369b022462 tempest-VolumesBackupsTest-615004682 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:09:19.103338 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-f4ecb8fc-be6f-4f03-a1c6-6c369b022462 tempest-VolumesBackupsTest-615004682 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.078s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:09:19.104479 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f4ecb8fc-be6f-4f03-a1c6-6c369b022462 tempest-VolumesBackupsTest-615004682 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-ab2daf87-1564-4764-b491-1e5192ed5de8', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'ab2daf87-1564-4764-b491-1e5192ed5de8', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:09:19.133009 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f4ecb8fc-be6f-4f03-a1c6-6c369b022462 tempest-VolumesBackupsTest-615004682 None] Initialize volume connection completed successfully. Sep 17 21:09:24.004276 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f4ecb8fc-be6f-4f03-a1c6-6c369b022462 tempest-VolumesBackupsTest-615004682 None] Terminate volume connection completed successfully. Sep 17 21:09:24.037039 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f4ecb8fc-be6f-4f03-a1c6-6c369b022462 tempest-VolumesBackupsTest-615004682 None] Remove volume export completed successfully. Sep 17 21:09:29.343389 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-fc776714-c9a1-4fed-a1c3-0dbcb5e3104c req-e209a176-f8d0-45c7-bacf-1efbaa0af3ae tempest-VolumesBackupsTest-615004682 None] Terminate volume connection completed successfully. Sep 17 21:09:29.347659 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-fc776714-c9a1-4fed-a1c3-0dbcb5e3104c req-e209a176-f8d0-45c7-bacf-1efbaa0af3ae tempest-VolumesBackupsTest-615004682 None] Deleting attachment 3fe0c97e-31e7-4a8e-94d8-383b15c040eb. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:09:31.913943 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=31 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:37.936510 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a67ddf3c-e299-4363-ba0d-bb38ef5cc0d5 tempest-VolumesBackupsTest-615004682 None] Acquiring lock "cinder-ab2daf87-1564-4764-b491-1e5192ed5de8-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:09:37.954077 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a67ddf3c-e299-4363-ba0d-bb38ef5cc0d5 tempest-VolumesBackupsTest-615004682 None] Lock "cinder-ab2daf87-1564-4764-b491-1e5192ed5de8-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:09:37.961773 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=22,cinder:UPDATE=6 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:09:38.047482 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a67ddf3c-e299-4363-ba0d-bb38ef5cc0d5 tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:09:39.163719 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a67ddf3c-e299-4363-ba0d-bb38ef5cc0d5 tempest-VolumesBackupsTest-615004682 None] Volume volume-ab2daf87-1564-4764-b491-1e5192ed5de8 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:09:39.205982 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a67ddf3c-e299-4363-ba0d-bb38ef5cc0d5 tempest-VolumesBackupsTest-615004682 None] deleting rbd volume volume-ab2daf87-1564-4764-b491-1e5192ed5de8 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:09:39.634701 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-a67ddf3c-e299-4363-ba0d-bb38ef5cc0d5 tempest-VolumesBackupsTest-615004682 None] Created reservations ['cb6363e9-6801-4de9-9c8a-ee1d66c9760c', '3a6adbb6-62b8-4566-9965-38cc39f7af7f', 'fcc57c38-cced-4b7a-8f0e-212d07a76327', 'bc8da782-a0bf-48ae-b44c-40d954098797'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:09:39.733603 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a67ddf3c-e299-4363-ba0d-bb38ef5cc0d5 tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:09:39.796712 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a67ddf3c-e299-4363-ba0d-bb38ef5cc0d5 tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:09:39.870857 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-a67ddf3c-e299-4363-ba0d-bb38ef5cc0d5 tempest-VolumesBackupsTest-615004682 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:09:39.876063 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a67ddf3c-e299-4363-ba0d-bb38ef5cc0d5 tempest-VolumesBackupsTest-615004682 None] Deleted volume successfully. Sep 17 21:09:39.902427 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a67ddf3c-e299-4363-ba0d-bb38ef5cc0d5 tempest-VolumesBackupsTest-615004682 None] Lock "cinder-ab2daf87-1564-4764-b491-1e5192ed5de8-delete_volume" released by "delete_volume" :: held 1.948s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:09:41.262056 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:09:41.296194 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Flow 'volume_create_manager' (bf7ced39-5edc-4845-b12c-920b3cf8ed65) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:09:41.309422 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d1e894c7-a55b-4146-b935-512b7cef2207) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:41.331840 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d1e894c7-a55b-4146-b935-512b7cef2207) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:09:41Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1395875576',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3a17771a-5b2a-4fab-a309-8ad68b2a890d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='42e1e6b096bb41419c594dcde97e16e7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:09:41Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:09:41Z,use_quota=True,user_id='6d03e73a3bed48f6bf61a702f0f41658',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:41.338332 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (762ada42-22d5-407d-8b94-7719bc216aaf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:41.344435 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (762ada42-22d5-407d-8b94-7719bc216aaf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:41.351262 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23e8c2cb-eef2-43d0-98b7-150c9fc456c9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:41.494882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23e8c2cb-eef2-43d0-98b7-150c9fc456c9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '3a17771a-5b2a-4fab-a309-8ad68b2a890d', 'volume_name': 'volume-3a17771a-5b2a-4fab-a309-8ad68b2a890d', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:41.501111 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d9b9f0cb-320f-4945-8ef2-bf739d22f7e2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:41.507953 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d9b9f0cb-320f-4945-8ef2-bf739d22f7e2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:41.515029 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f62f05b2-c4df-407e-bdcf-2bfd856dbc95) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:41.516811 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Volume 3a17771a-5b2a-4fab-a309-8ad68b2a890d: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-3a17771a-5b2a-4fab-a309-8ad68b2a890d', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 21:09:41.517898 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Cloning 3a17771a-5b2a-4fab-a309-8ad68b2a890d from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 21:09:41.518833 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:09:41.531863 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.013s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:09:41.550543 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 9, 41, 541914)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:09:41.551701 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:09:41.552735 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 21:09:41.563187 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.031s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:09:41.564683 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 21:09:41.573534 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 9, 41, 569981)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:09:41.574158 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:09:41.575249 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 21:09:41.596857 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:09:41.731741 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] creating snapshot='volume-3a17771a-5b2a-4fab-a309-8ad68b2a890d.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:09:42.248632 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-3a17771a-5b2a-4fab-a309-8ad68b2a890d.clone_snap' to 'volume-3a17771a-5b2a-4fab-a309-8ad68b2a890d' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:09:42.467224 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:09:42.469343 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:09:42.469953 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:09:42.500786 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:09:42.502118 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Marking volume 3a17771a-5b2a-4fab-a309-8ad68b2a890d as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:09:42.516613 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to 3a17771a-5b2a-4fab-a309-8ad68b2a890d. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 21:09:42.517235 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Creating volume glance metadata for volume 3a17771a-5b2a-4fab-a309-8ad68b2a890d backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 21:09:42.614431 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f62f05b2-c4df-407e-bdcf-2bfd856dbc95) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3a17771a-5b2a-4fab-a309-8ad68b2a890d', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:42.620371 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (002e6972-8c00-4182-922f-515adeffa52c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:42.642095 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Volume volume-3a17771a-5b2a-4fab-a309-8ad68b2a890d (3a17771a-5b2a-4fab-a309-8ad68b2a890d): created successfully Sep 17 21:09:42.648702 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (002e6972-8c00-4182-922f-515adeffa52c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:42.657066 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Flow 'volume_create_manager' (bf7ced39-5edc-4845-b12c-920b3cf8ed65) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:09:42.681012 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5e0b3b48-736f-4a9c-ae2c-67918958ccc5 tempest-VolumesBackupsTest-615004682 None] Created volume successfully. Sep 17 21:09:43.723992 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b79073e7-7a84-4fae-b7cf-822d0e5e80dc tempest-VolumesBackupsTest-615004682 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-17T21:09:43Z,data_timestamp=2022-09-17T21:09:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-205221736',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=b1236513-f501-4dff-a597-e3bee98ea41f,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='42e1e6b096bb41419c594dcde97e16e7',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='6d03e73a3bed48f6bf61a702f0f41658',volume_id=3a17771a-5b2a-4fab-a309-8ad68b2a890d) Sep 17 21:09:43.725094 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.backup.rpcapi [None req-b79073e7-7a84-4fae-b7cf-822d0e5e80dc tempest-VolumesBackupsTest-615004682 None] continue_backup in rpcapi backup_id b1236513-f501-4dff-a597-e3bee98ea41f {{(pid=133409) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 17 21:09:43.892302 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-b79073e7-7a84-4fae-b7cf-822d0e5e80dc tempest-VolumesBackupsTest-615004682 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:09:44.092932 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-e38e1ab8-8963-4253-822f-4abe43a24cde req-64ac2a63-476f-45c1-a915-a13b49ed33e3 tempest-ServerRescueNegativeTestJSON-533169750 None] Terminate volume connection completed successfully. Sep 17 21:09:44.093404 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e38e1ab8-8963-4253-822f-4abe43a24cde req-64ac2a63-476f-45c1-a915-a13b49ed33e3 tempest-ServerRescueNegativeTestJSON-533169750 None] Deleting attachment a630e7ce-6cdd-4e32-969b-b753303919ab. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:09:45.313152 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-b79073e7-7a84-4fae-b7cf-822d0e5e80dc tempest-VolumesBackupsTest-615004682 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.422s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:09:45.314006 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b79073e7-7a84-4fae-b7cf-822d0e5e80dc tempest-VolumesBackupsTest-615004682 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-3a17771a-5b2a-4fab-a309-8ad68b2a890d', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '3a17771a-5b2a-4fab-a309-8ad68b2a890d', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:09:45.341719 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b79073e7-7a84-4fae-b7cf-822d0e5e80dc tempest-VolumesBackupsTest-615004682 None] Initialize volume connection completed successfully. Sep 17 21:09:55.831050 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b79073e7-7a84-4fae-b7cf-822d0e5e80dc tempest-VolumesBackupsTest-615004682 None] Terminate volume connection completed successfully. Sep 17 21:09:55.878778 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b79073e7-7a84-4fae-b7cf-822d0e5e80dc tempest-VolumesBackupsTest-615004682 None] Remove volume export completed successfully. Sep 17 21:09:57.259839 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:09:57.312450 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Flow 'volume_create_manager' (4598350e-0f1c-43e9-a190-4dac6091a835) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:09:57.331465 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cfb2ae46-aa29-4e03-8513-4469c135f538) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:57.379851 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cfb2ae46-aa29-4e03-8513-4469c135f538) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:09:57Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_b1236513-f501-4dff-a597-e3bee98ea41f',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=12100b1a-0149-4451-a823-a3d14e132e07,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='42e1e6b096bb41419c594dcde97e16e7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:09:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:09:57Z,use_quota=True,user_id='6d03e73a3bed48f6bf61a702f0f41658',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:57.399284 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (954ee8c8-74cd-4a17-844b-9e65e15efbc7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:57.404391 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (954ee8c8-74cd-4a17-844b-9e65e15efbc7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:57.409408 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7629071a-7390-491f-9eaa-b35af74c68d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:57.422052 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7629071a-7390-491f-9eaa-b35af74c68d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '12100b1a-0149-4451-a823-a3d14e132e07', 'volume_name': 'volume-12100b1a-0149-4451-a823-a3d14e132e07', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:57.430518 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (eeb69fb7-0094-4148-82a8-4f4a4b2d5813) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:57.449957 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (eeb69fb7-0094-4148-82a8-4f4a4b2d5813) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:57.455360 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca4ab002-7e35-4684-9b5f-c1d2b129a325) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:57.460116 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Volume 12100b1a-0149-4451-a823-a3d14e132e07: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-12100b1a-0149-4451-a823-a3d14e132e07', 'volume_size': 1} Sep 17 21:09:57.461074 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] creating volume 'volume-12100b1a-0149-4451-a823-a3d14e132e07' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:09:57.461972 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:09:57.636547 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:09:57.637009 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:09:57.640883 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca4ab002-7e35-4684-9b5f-c1d2b129a325) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-12100b1a-0149-4451-a823-a3d14e132e07', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:57.650493 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (19f963ea-450d-4abb-9214-73e25ba64703) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:57.676712 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Volume volume-12100b1a-0149-4451-a823-a3d14e132e07 (12100b1a-0149-4451-a823-a3d14e132e07): created successfully Sep 17 21:09:57.680923 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (19f963ea-450d-4abb-9214-73e25ba64703) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:57.694429 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Flow 'volume_create_manager' (4598350e-0f1c-43e9-a190-4dac6091a835) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:09:57.711654 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Created volume successfully. Sep 17 21:09:58.403524 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:09:59.060877 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] Acquiring lock "cinder-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:09:59.091102 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] Lock "cinder-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa-delete_volume" acquired by "delete_volume" :: waited 0.030s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:09:59.216118 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:09:59.352315 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:09:59.352758 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] Volume volume-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:09:59.370820 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] deleting rbd volume volume-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:09:59.728557 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:09:59.761260 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Flow 'volume_create_manager' (2a6c6eca-fa33-4dad-a3cb-9f925b305aba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:09:59.772421 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fc1928c3-c3ce-4db6-95cd-90fc0622c2d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:59.816016 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fc1928c3-c3ce-4db6-95cd-90fc0622c2d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:09:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-1077347474',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f04b4403-7625-47e5-a148-a05fd9ddc8e4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6fa7e14863254ab3964ac907c168d9e6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:10:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:10:00Z,use_quota=True,user_id='c544e8e6f7a741c985c92ed7437397cd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:59.825397 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (56f6258f-8b65-481d-a18d-a694cc778c94) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:59.838374 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (56f6258f-8b65-481d-a18d-a694cc778c94) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:59.842802 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (037b11c1-db66-4537-9d99-ecde7355e7d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:59.847401 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (037b11c1-db66-4537-9d99-ecde7355e7d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f04b4403-7625-47e5-a148-a05fd9ddc8e4', 'volume_name': 'volume-f04b4403-7625-47e5-a148-a05fd9ddc8e4', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:59.855744 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8287d41a-ad91-4d3b-9f17-fd17486c3d5c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:59.868158 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8287d41a-ad91-4d3b-9f17-fd17486c3d5c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:09:59.876489 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (41c9a882-0bfe-46cb-aac3-b7a2b2a8e032) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:09:59.878314 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Volume f04b4403-7625-47e5-a148-a05fd9ddc8e4: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f04b4403-7625-47e5-a148-a05fd9ddc8e4', 'volume_size': 1} Sep 17 21:09:59.879063 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] creating volume 'volume-f04b4403-7625-47e5-a148-a05fd9ddc8e4' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:09:59.880136 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:09:59.972905 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.569s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:09:59.979695 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-12100b1a-0149-4451-a823-a3d14e132e07', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '12100b1a-0149-4451-a823-a3d14e132e07', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:10:00.013559 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Initialize volume connection completed successfully. Sep 17 21:10:00.096811 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:10:00.098041 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:10:00.103481 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (41c9a882-0bfe-46cb-aac3-b7a2b2a8e032) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f04b4403-7625-47e5-a148-a05fd9ddc8e4', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:00.118328 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (37a325b2-c528-42f1-9d9a-036c18d1d53a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:00.184360 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Volume volume-f04b4403-7625-47e5-a148-a05fd9ddc8e4 (f04b4403-7625-47e5-a148-a05fd9ddc8e4): created successfully Sep 17 21:10:00.188873 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (37a325b2-c528-42f1-9d9a-036c18d1d53a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:00.196912 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Flow 'volume_create_manager' (2a6c6eca-fa33-4dad-a3cb-9f925b305aba) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:10:00.238871 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-360e9444-0339-491c-a892-dd01c1059808 tempest-AttachVolumeNegativeTest-1020448628 None] Created volume successfully. Sep 17 21:10:00.262758 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] Created reservations ['58f7cfb8-8b9a-4d1a-8fbb-27cd21037aa4', '26d796c9-baf6-4e68-a67e-bf539be65817', 'c5259a24-eb5d-4146-b0a6-f20384e425cd', 'b7e26ed2-66a4-4bf3-8d54-61697eeb1a1d'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:10:00.434682 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=63,cinder:UPDATE=23,cinder:INSERT=12,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:00.491592 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:00.564696 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:00.599018 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:10:00.606410 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] Deleted volume successfully. Sep 17 21:10:00.627528 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-092e0b0e-0b3c-4b84-961f-5c14abc5d456 tempest-ServerRescueNegativeTestJSON-533169750 None] Lock "cinder-5ba7aae4-11cf-4bae-825a-6ea9f7b052fa-delete_volume" released by "delete_volume" :: held 1.536s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:10:01.035787 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:10:01.039055 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:01.113646 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:01.196912 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:10:01.618316 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] Acquiring lock "cinder-f337f44a-3587-4bc6-933a-9287690ea825-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:10:01.661068 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] Lock "cinder-f337f44a-3587-4bc6-933a-9287690ea825-delete_volume" acquired by "delete_volume" :: waited 0.043s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:10:01.800438 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:02.048910 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:10:02.049701 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] Volume volume-f337f44a-3587-4bc6-933a-9287690ea825 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:10:02.097587 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] deleting rbd volume volume-f337f44a-3587-4bc6-933a-9287690ea825 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:10:03.452768 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-018f30e9-4681-4902-a5d6-81a11f9ef24d req-cb12b0ab-7bf7-47e3-aa0c-4690c91a07d2 tempest-AttachVolumeNegativeTest-1020448628 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:10:04.717075 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=67,cinder:UPDATE=22,cinder:DELETE=3,cinder:INSERT=8 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:04.768032 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] Created reservations ['4d65afda-72f3-4f90-a9aa-4cba65e0ef44', 'b13437e3-1edb-469c-b430-296b7bce49a4', '18fe6e89-e723-4374-8570-12d4987dd4c5', 'ea3aea45-b4d0-4b15-be83-566bdc858c36'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:10:04.891299 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:05.001961 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:05.086689 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:10:05.126779 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] Deleted volume successfully. Sep 17 21:10:05.211398 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b2add720-edfd-41ad-931a-230742c27dfc tempest-ServerRescueNegativeTestJSON-533169750 None] Lock "cinder-f337f44a-3587-4bc6-933a-9287690ea825-delete_volume" released by "delete_volume" :: held 3.550s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:10:05.975652 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-018f30e9-4681-4902-a5d6-81a11f9ef24d req-cb12b0ab-7bf7-47e3-aa0c-4690c91a07d2 tempest-AttachVolumeNegativeTest-1020448628 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 2.523s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:10:05.976586 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-018f30e9-4681-4902-a5d6-81a11f9ef24d req-cb12b0ab-7bf7-47e3-aa0c-4690c91a07d2 tempest-AttachVolumeNegativeTest-1020448628 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f04b4403-7625-47e5-a148-a05fd9ddc8e4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'f04b4403-7625-47e5-a148-a05fd9ddc8e4', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:10:06.023484 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-018f30e9-4681-4902-a5d6-81a11f9ef24d req-cb12b0ab-7bf7-47e3-aa0c-4690c91a07d2 tempest-AttachVolumeNegativeTest-1020448628 None] Connection info returned from driver {'name': 'volumes/volume-f04b4403-7625-47e5-a148-a05fd9ddc8e4', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'f04b4403-7625-47e5-a148-a05fd9ddc8e4', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '9acd188c-57d3-4c78-840e-a202491dc828'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:10:06.166755 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-018f30e9-4681-4902-a5d6-81a11f9ef24d req-cb12b0ab-7bf7-47e3-aa0c-4690c91a07d2 tempest-AttachVolumeNegativeTest-1020448628 None] attachment_update completed successfully. Sep 17 21:10:10.168243 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Terminate volume connection completed successfully. Sep 17 21:10:10.190871 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b09dc098-cee8-4829-83e7-977b6b2c8f2a tempest-VolumesBackupsTest-615004682 None] Remove volume export completed successfully. Sep 17 21:10:10.745165 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-d35fd16f-d1e7-4c9c-ba60-8659adacab70 req-472a9fa5-00ef-4cc9-9d71-0ea089c69ee3 tempest-AttachVolumeNegativeTest-1020448628 None] Terminate volume connection completed successfully. Sep 17 21:10:10.759660 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-d35fd16f-d1e7-4c9c-ba60-8659adacab70 req-472a9fa5-00ef-4cc9-9d71-0ea089c69ee3 tempest-AttachVolumeNegativeTest-1020448628 None] Deleting attachment 9acd188c-57d3-4c78-840e-a202491dc828. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:10:11.448801 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3180a93a-2bcb-483c-9e7a-106110180d7f tempest-VolumesBackupsTest-615004682 None] Acquiring lock "cinder-12100b1a-0149-4451-a823-a3d14e132e07-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:10:11.495643 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3180a93a-2bcb-483c-9e7a-106110180d7f tempest-VolumesBackupsTest-615004682 None] Lock "cinder-12100b1a-0149-4451-a823-a3d14e132e07-delete_volume" acquired by "delete_volume" :: waited 0.047s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:10:11.589761 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3180a93a-2bcb-483c-9e7a-106110180d7f tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:12.353021 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3180a93a-2bcb-483c-9e7a-106110180d7f tempest-VolumesBackupsTest-615004682 None] Volume volume-12100b1a-0149-4451-a823-a3d14e132e07 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:10:12.397716 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3180a93a-2bcb-483c-9e7a-106110180d7f tempest-VolumesBackupsTest-615004682 None] deleting rbd volume volume-12100b1a-0149-4451-a823-a3d14e132e07 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:10:12.923558 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-3180a93a-2bcb-483c-9e7a-106110180d7f tempest-VolumesBackupsTest-615004682 None] Created reservations ['369b7058-916b-4c34-8f17-e9a4a5deac63', 'cec5c75f-4db0-40c7-8025-30102a19cbab', 'c314bf7b-95ab-40e5-9ee2-ef060a1dad4c', '345293fe-6288-4ab6-8bf8-67baf9eae0de'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:10:12.973184 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3180a93a-2bcb-483c-9e7a-106110180d7f tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:13.000146 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3180a93a-2bcb-483c-9e7a-106110180d7f tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:13.034836 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3180a93a-2bcb-483c-9e7a-106110180d7f tempest-VolumesBackupsTest-615004682 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:10:13.054020 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3180a93a-2bcb-483c-9e7a-106110180d7f tempest-VolumesBackupsTest-615004682 None] Deleted volume successfully. Sep 17 21:10:13.072992 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3180a93a-2bcb-483c-9e7a-106110180d7f tempest-VolumesBackupsTest-615004682 None] Lock "cinder-12100b1a-0149-4451-a823-a3d14e132e07-delete_volume" released by "delete_volume" :: held 1.577s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:10:15.970377 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] Acquiring lock "cinder-3a17771a-5b2a-4fab-a309-8ad68b2a890d-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:10:15.986924 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] Lock "cinder-3a17771a-5b2a-4fab-a309-8ad68b2a890d-delete_volume" acquired by "delete_volume" :: waited 0.017s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:10:16.055431 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:16.533152 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] deleting rbd volume volume-3a17771a-5b2a-4fab-a309-8ad68b2a890d {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:10:16.971921 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:10:17.018095 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] deleting parent snapshot volume-3a17771a-5b2a-4fab-a309-8ad68b2a890d.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:10:17.623267 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] Created reservations ['374406cb-a0db-4d20-bf17-28357401455f', '56c2c41c-1122-4ce5-a5cd-fcb1ae2a71e9', 'ccc397bd-0c93-400c-aa8f-866eda0d924a', 'a19cbbcb-37d1-4c57-8683-6203ce36a87b'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:10:17.725263 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:17.801308 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:17.882480 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:10:17.899338 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] Deleted volume successfully. Sep 17 21:10:17.929566 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ebbc8aed-da5f-4cf7-82fd-a8ebf182e954 tempest-VolumesBackupsTest-615004682 None] Lock "cinder-3a17771a-5b2a-4fab-a309-8ad68b2a890d-delete_volume" released by "delete_volume" :: held 1.942s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:10:21.552117 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:10:21.574543 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Flow 'volume_create_manager' (b62cc1a3-76b1-40f9-a0e9-df10e3ed0d56) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:10:21.581887 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a742bb32-717f-49b8-9edb-d3fc4286e951) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:21.599370 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a742bb32-717f-49b8-9edb-d3fc4286e951) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:10:21Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Volume-1886996017',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f9fc498a-64cc-497e-99c0-eecfbbd5e9e3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0a2922eb9e0245c19efd8ff93adf68a9',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:10:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:10:22Z,use_quota=True,user_id='224b008680cf4312a5a4bec862e86960',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:21.604578 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (48d85bd5-4b34-40b0-b68f-710b548df0d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:21.609229 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (48d85bd5-4b34-40b0-b68f-710b548df0d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:21.614433 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e10bc104-598f-4ccf-b8c7-b8bacd9d665a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:21.618916 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e10bc104-598f-4ccf-b8c7-b8bacd9d665a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f9fc498a-64cc-497e-99c0-eecfbbd5e9e3', 'volume_name': 'volume-f9fc498a-64cc-497e-99c0-eecfbbd5e9e3', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:21.624358 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1e2e510b-5da4-4f00-98ef-cd9753535bc3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:21.628752 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1e2e510b-5da4-4f00-98ef-cd9753535bc3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:21.633831 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dcdac962-f62b-4067-b245-4d4ad7dcaa72) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:21.635132 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Volume f9fc498a-64cc-497e-99c0-eecfbbd5e9e3: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f9fc498a-64cc-497e-99c0-eecfbbd5e9e3', 'volume_size': 1} Sep 17 21:10:21.641961 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] creating volume 'volume-f9fc498a-64cc-497e-99c0-eecfbbd5e9e3' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:10:21.642959 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:21.737659 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:10:21.738096 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:10:21.742725 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dcdac962-f62b-4067-b245-4d4ad7dcaa72) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f9fc498a-64cc-497e-99c0-eecfbbd5e9e3', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:21.748536 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8d5a4227-cf15-4d9d-bdf2-f40e168e947b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:21.762412 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Volume volume-f9fc498a-64cc-497e-99c0-eecfbbd5e9e3 (f9fc498a-64cc-497e-99c0-eecfbbd5e9e3): created successfully Sep 17 21:10:21.768224 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8d5a4227-cf15-4d9d-bdf2-f40e168e947b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:21.775362 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Flow 'volume_create_manager' (b62cc1a3-76b1-40f9-a0e9-df10e3ed0d56) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:10:21.788957 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7cb1f79f-8cc1-438a-bfd7-f04cd6866d24 tempest-VolumesBackupsV39Test-147641996 None] Created volume successfully. Sep 17 21:10:22.835265 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-933d102a-10e5-4b67-af04-63bb6600fe18 tempest-VolumesBackupsV39Test-147641996 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-17T21:10:22Z,data_timestamp=2022-09-17T21:10:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Backup-727964005',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=16abdd73-6c5d-4f19-814c-ae603935c4e6,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='0a2922eb9e0245c19efd8ff93adf68a9',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='224b008680cf4312a5a4bec862e86960',volume_id=f9fc498a-64cc-497e-99c0-eecfbbd5e9e3) Sep 17 21:10:22.836788 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.backup.rpcapi [None req-933d102a-10e5-4b67-af04-63bb6600fe18 tempest-VolumesBackupsV39Test-147641996 None] continue_backup in rpcapi backup_id 16abdd73-6c5d-4f19-814c-ae603935c4e6 {{(pid=133409) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 17 21:10:23.034288 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-933d102a-10e5-4b67-af04-63bb6600fe18 tempest-VolumesBackupsV39Test-147641996 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:10:24.661683 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-933d102a-10e5-4b67-af04-63bb6600fe18 tempest-VolumesBackupsV39Test-147641996 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.628s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:10:24.662643 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-933d102a-10e5-4b67-af04-63bb6600fe18 tempest-VolumesBackupsV39Test-147641996 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f9fc498a-64cc-497e-99c0-eecfbbd5e9e3', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'f9fc498a-64cc-497e-99c0-eecfbbd5e9e3', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:10:24.693556 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-933d102a-10e5-4b67-af04-63bb6600fe18 tempest-VolumesBackupsV39Test-147641996 None] Initialize volume connection completed successfully. Sep 17 21:10:26.396012 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-933d102a-10e5-4b67-af04-63bb6600fe18 tempest-VolumesBackupsV39Test-147641996 None] Terminate volume connection completed successfully. Sep 17 21:10:26.432152 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-933d102a-10e5-4b67-af04-63bb6600fe18 tempest-VolumesBackupsV39Test-147641996 None] Remove volume export completed successfully. Sep 17 21:10:29.651585 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9079ff65-fb96-4261-a50e-9b9a784958a8 tempest-VolumesBackupsV39Test-147641996 None] Acquiring lock "cinder-f9fc498a-64cc-497e-99c0-eecfbbd5e9e3-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:10:29.666512 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9079ff65-fb96-4261-a50e-9b9a784958a8 tempest-VolumesBackupsV39Test-147641996 None] Lock "cinder-f9fc498a-64cc-497e-99c0-eecfbbd5e9e3-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:10:29.679915 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=28,cinder:SELECT=64,cinder:INSERT=7,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:29.723031 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=31,cinder:SELECT=61,cinder:DELETE=3,cinder:INSERT=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:29.729836 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9079ff65-fb96-4261-a50e-9b9a784958a8 tempest-VolumesBackupsV39Test-147641996 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:30.300736 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9079ff65-fb96-4261-a50e-9b9a784958a8 tempest-VolumesBackupsV39Test-147641996 None] Volume volume-f9fc498a-64cc-497e-99c0-eecfbbd5e9e3 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:10:30.336125 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9079ff65-fb96-4261-a50e-9b9a784958a8 tempest-VolumesBackupsV39Test-147641996 None] deleting rbd volume volume-f9fc498a-64cc-497e-99c0-eecfbbd5e9e3 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:10:30.854014 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-9079ff65-fb96-4261-a50e-9b9a784958a8 tempest-VolumesBackupsV39Test-147641996 None] Created reservations ['f6fd2d77-54ea-44a3-820e-f1860527a3ad', '8744b6f1-c0fe-427d-8b75-a96540e9ef7d', 'd8a5d517-c115-4948-bb84-62671fee65f8', 'e065fba4-8e5f-4fc1-af41-14c0159f6768'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:10:31.005330 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9079ff65-fb96-4261-a50e-9b9a784958a8 tempest-VolumesBackupsV39Test-147641996 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:31.070696 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9079ff65-fb96-4261-a50e-9b9a784958a8 tempest-VolumesBackupsV39Test-147641996 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:31.181284 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-9079ff65-fb96-4261-a50e-9b9a784958a8 tempest-VolumesBackupsV39Test-147641996 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:10:31.187902 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-9079ff65-fb96-4261-a50e-9b9a784958a8 tempest-VolumesBackupsV39Test-147641996 None] Deleted volume successfully. Sep 17 21:10:31.216232 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9079ff65-fb96-4261-a50e-9b9a784958a8 tempest-VolumesBackupsV39Test-147641996 None] Lock "cinder-f9fc498a-64cc-497e-99c0-eecfbbd5e9e3-delete_volume" released by "delete_volume" :: held 1.549s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:10:37.369975 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:10:37.424735 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Flow 'volume_create_manager' (c13272b6-111c-4853-a902-dada3b1adddd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:10:37.442410 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (87fdb0e7-6197-4022-861c-be6ea7bbb63a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:37.467161 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (87fdb0e7-6197-4022-861c-be6ea7bbb63a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:10:37Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesExtendTest-Volume-2142224131',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ab0ae691-8edb-40ce-ac9c-b932a46d8cdc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='93597f97e50a47eb9e718a168a11dad1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:10:37Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:10:37Z,use_quota=True,user_id='1c225ad7f31942838eb1391a988b59ab',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:37.496968 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d0de04aa-611c-430d-96f5-760357a02cc3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:37.510083 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d0de04aa-611c-430d-96f5-760357a02cc3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:37.522516 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e54c9320-f71e-45e4-b635-5a7b5ad46dd5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:37.710634 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e54c9320-f71e-45e4-b635-5a7b5ad46dd5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'ab0ae691-8edb-40ce-ac9c-b932a46d8cdc', 'volume_name': 'volume-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:37.724153 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (05e16ca4-e824-4141-a68a-84e11fdf6b18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:37.741802 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (05e16ca4-e824-4141-a68a-84e11fdf6b18) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:37.746882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (84d34a31-f6c1-44ac-a77f-6f66ac56b9a0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:37.748100 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Volume ab0ae691-8edb-40ce-ac9c-b932a46d8cdc: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 21:10:37.748656 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Cloning ab0ae691-8edb-40ce-ac9c-b932a46d8cdc from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 21:10:37.749614 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:10:37.765670 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:10:37.804124 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 10, 37, 777114)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:10:37.804942 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:10:37.805742 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 21:10:37.817752 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.046s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:10:37.819173 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 21:10:37.836379 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 10, 37, 827252)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:10:37.836810 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:10:37.837202 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 21:10:37.864758 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:37.974659 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] creating snapshot='volume-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:10:38.512449 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc.clone_snap' to 'volume-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:10:38.758935 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:10:38.760190 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:10:38.760551 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:10:38.784561 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:10:38.785540 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Marking volume ab0ae691-8edb-40ce-ac9c-b932a46d8cdc as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:10:38.795861 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to ab0ae691-8edb-40ce-ac9c-b932a46d8cdc. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 21:10:38.796456 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Creating volume glance metadata for volume ab0ae691-8edb-40ce-ac9c-b932a46d8cdc backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 21:10:38.876944 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (84d34a31-f6c1-44ac-a77f-6f66ac56b9a0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:38.887730 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1d6cd476-d8c3-478b-859c-90536ef15846) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:38.908300 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Volume volume-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc (ab0ae691-8edb-40ce-ac9c-b932a46d8cdc): created successfully Sep 17 21:10:38.918462 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1d6cd476-d8c3-478b-859c-90536ef15846) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:38.931471 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Flow 'volume_create_manager' (c13272b6-111c-4853-a902-dada3b1adddd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:10:38.954111 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-08a8b5f1-1e5f-4b9c-91de-87a4f54a940d tempest-VolumesExtendTest-2064453780 None] Created volume successfully. Sep 17 21:10:39.647221 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-215a32f3-b68c-44df-b13a-1677a5661be6 tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:39.949424 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-215a32f3-b68c-44df-b13a-1677a5661be6 tempest-VolumesExtendTest-2064453780 None] Extend volume from 1 GB to 2 GB. {{(pid=133409) extend_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:2008}} Sep 17 21:10:40.031360 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-215a32f3-b68c-44df-b13a-1677a5661be6 tempest-VolumesExtendTest-2064453780 None] Extend volume completed successfully. Sep 17 21:10:41.587924 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:10:41.628489 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Flow 'volume_create_manager' (838ec0b7-16d5-41be-8c2c-27eed7bcf7ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:10:41.647795 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (09003535-510c-49d7-bce8-bb964e3467ec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:41.687102 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (09003535-510c-49d7-bce8-bb964e3467ec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:10:41Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesExtendTest-Volume-1324445474',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8769fcb9-f095-4a01-8905-1f2146923e36,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='93597f97e50a47eb9e718a168a11dad1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:10:41Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:10:42Z,use_quota=True,user_id='1c225ad7f31942838eb1391a988b59ab',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:41.702165 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7ddda000-3cf4-4025-bffe-0f4880b0297c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:41.728235 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7ddda000-3cf4-4025-bffe-0f4880b0297c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:41.742501 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1485d68c-e937-4f8f-9b66-c1a095888dff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:41.754480 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1485d68c-e937-4f8f-9b66-c1a095888dff) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8769fcb9-f095-4a01-8905-1f2146923e36', 'volume_name': 'volume-8769fcb9-f095-4a01-8905-1f2146923e36', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:41.770408 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2272dc71-b964-4e63-8da0-04df302b89ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:41.803325 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2272dc71-b964-4e63-8da0-04df302b89ba) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:41.822016 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7fb6f06a-41fa-47f7-ac1a-05580e67b13a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:41.824768 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Volume 8769fcb9-f095-4a01-8905-1f2146923e36: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8769fcb9-f095-4a01-8905-1f2146923e36', 'volume_size': 1} Sep 17 21:10:41.826183 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] creating volume 'volume-8769fcb9-f095-4a01-8905-1f2146923e36' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:10:41.827369 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:41.991098 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:10:41.991752 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:10:41.995957 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7fb6f06a-41fa-47f7-ac1a-05580e67b13a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8769fcb9-f095-4a01-8905-1f2146923e36', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:42.007027 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9baf2b69-5534-4f7d-b587-7c365774c563) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:42.027191 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Volume volume-8769fcb9-f095-4a01-8905-1f2146923e36 (8769fcb9-f095-4a01-8905-1f2146923e36): created successfully Sep 17 21:10:42.031227 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9baf2b69-5534-4f7d-b587-7c365774c563) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:42.040838 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Flow 'volume_create_manager' (838ec0b7-16d5-41be-8c2c-27eed7bcf7ff) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:10:42.060809 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-dc58b6e5-9221-4413-a121-7ccb42bdb170 tempest-VolumesExtendTest-2064453780 None] Created volume successfully. Sep 17 21:10:43.110530 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-77fac9bb-f99e-4a23-a6db-42aaba6af6ae tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:43.864107 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-77fac9bb-f99e-4a23-a6db-42aaba6af6ae tempest-VolumesExtendTest-2064453780 None] Create snapshot completed successfully Sep 17 21:10:44.663129 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e15b6ee7-10fb-41cb-896a-2ed22038095c tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:44.919113 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e15b6ee7-10fb-41cb-896a-2ed22038095c tempest-VolumesExtendTest-2064453780 None] Extend volume from 1 GB to 2 GB. {{(pid=133409) extend_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:2008}} Sep 17 21:10:44.978971 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e15b6ee7-10fb-41cb-896a-2ed22038095c tempest-VolumesExtendTest-2064453780 None] Extend volume completed successfully. Sep 17 21:10:46.138855 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-219983f1-9fb1-41b7-a66c-a52549339692 tempest-VolumesExtendTest-2064453780 None] Acquiring lock "cinder-5a623145-39f2-472c-8aa9-f37ada302515-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:10:46.160309 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-219983f1-9fb1-41b7-a66c-a52549339692 tempest-VolumesExtendTest-2064453780 None] Lock "cinder-5a623145-39f2-472c-8aa9-f37ada302515-delete_snapshot" acquired by "delete_snapshot" :: waited 0.022s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:10:46.171904 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-219983f1-9fb1-41b7-a66c-a52549339692 tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:47.265402 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-219983f1-9fb1-41b7-a66c-a52549339692 tempest-VolumesExtendTest-2064453780 None] Deprecated: Default quota for resource: snapshots_ceph is set by the default quota flag: quota_snapshots_ceph, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:10:47.286229 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-219983f1-9fb1-41b7-a66c-a52549339692 tempest-VolumesExtendTest-2064453780 None] Created reservations ['f5fb9eff-0661-4698-a0ba-2778246e81f3', '481413d8-2c56-47a1-8201-9c5f03634569', '4914dc3e-5b04-4bf6-8b2e-5abf0b6052e6', '8ff4ca86-e5ad-4d33-92f6-2a267666c747'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:10:47.378150 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-219983f1-9fb1-41b7-a66c-a52549339692 tempest-VolumesExtendTest-2064453780 None] Delete snapshot completed successfully. Sep 17 21:10:47.385415 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-219983f1-9fb1-41b7-a66c-a52549339692 tempest-VolumesExtendTest-2064453780 None] Lock "cinder-5a623145-39f2-472c-8aa9-f37ada302515-delete_snapshot" released by "delete_snapshot" :: held 1.225s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:10:48.485192 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] Acquiring lock "cinder-8769fcb9-f095-4a01-8905-1f2146923e36-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:10:48.525977 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] Lock "cinder-8769fcb9-f095-4a01-8905-1f2146923e36-delete_volume" acquired by "delete_volume" :: waited 0.040s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:10:48.579497 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=54,cinder:INSERT=15,cinder:UPDATE=28,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:48.618602 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:48.760477 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:10:48.761381 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] Volume volume-8769fcb9-f095-4a01-8905-1f2146923e36 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:10:48.772805 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] deleting rbd volume volume-8769fcb9-f095-4a01-8905-1f2146923e36 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:10:49.557879 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] Created reservations ['d0e19dc6-2350-42e1-92f7-6877a3f137b5', 'c4d72b6e-5642-4ae3-a165-1debb4a1ca87', 'a761ac03-4d53-42df-ba0a-b85a006ca4b8', '2a2aec1c-9a6b-49cc-99ec-0886b339fe8c'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:10:49.723031 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:49.840713 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] Acquiring lock "cinder-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:10:49.843937 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:49.941373 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:10:49.955418 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] Deleted volume successfully. Sep 17 21:10:49.958986 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=61,cinder:INSERT=7,cinder:UPDATE=31,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:10:49.982982 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] Lock "cinder-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc-delete_volume" acquired by "delete_volume" :: waited 0.142s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:10:50.017023 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-03ca0dd2-9d2b-4996-a3fd-6de5cc51f1af tempest-VolumesExtendTest-2064453780 None] Lock "cinder-8769fcb9-f095-4a01-8905-1f2146923e36-delete_volume" released by "delete_volume" :: held 1.488s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:10:50.080888 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:50.214926 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:10:50.234478 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] deleting rbd volume volume-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:10:50.817006 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:10:50.864899 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] deleting parent snapshot volume-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:10:51.400435 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] Created reservations ['af065d85-a5a0-47ec-a9d4-1dfe21024ae4', '45d4ce3b-77d2-4ac2-a844-16900853d125', 'b83e6b05-b252-4296-8823-f3502e654613', 'cb8c3373-1279-42bd-8168-efcd9d81e146'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:10:51.546184 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:51.631642 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:51.734706 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:10:51.746795 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] Deleted volume successfully. Sep 17 21:10:51.784119 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3ae0a27c-dc43-4534-98e2-e2dd2903b9e7 tempest-VolumesExtendTest-2064453780 None] Lock "cinder-ab0ae691-8edb-40ce-ac9c-b932a46d8cdc-delete_volume" released by "delete_volume" :: held 1.801s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:10:56.905357 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:10:56.931307 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Flow 'volume_create_manager' (6affacc0-8aa4-4fa0-bbcc-4c6b40193a2f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:10:56.940799 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6018ec9c-1c94-42dc-a60c-51b73cb39504) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:56.982066 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6018ec9c-1c94-42dc-a60c-51b73cb39504) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:10:57Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesNegativeTest-Volume-1445499312',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8d910032-95a8-4e99-9b61-7f91786325a8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9818fed65901412b8da672532cc5bc52',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:10:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:10:57Z,use_quota=True,user_id='7e12ad7a93ae4a9a95b2a54c1a2b9845',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:56.997805 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8337a030-b536-40fa-a5e0-2ff63778f620) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:57.014048 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8337a030-b536-40fa-a5e0-2ff63778f620) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:57.022197 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (810e0417-a08b-42ee-a788-bdfda5dcd5b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:57.027247 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (810e0417-a08b-42ee-a788-bdfda5dcd5b1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8d910032-95a8-4e99-9b61-7f91786325a8', 'volume_name': 'volume-8d910032-95a8-4e99-9b61-7f91786325a8', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:57.034682 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d8412bde-2cb7-4b2c-a8d0-3e476d3d1d3c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:57.039703 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d8412bde-2cb7-4b2c-a8d0-3e476d3d1d3c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:57.045617 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3dc2b630-ad99-4fae-a4e9-811af8702b12) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:57.047050 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Volume 8d910032-95a8-4e99-9b61-7f91786325a8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8d910032-95a8-4e99-9b61-7f91786325a8', 'volume_size': 1} Sep 17 21:10:57.047688 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] creating volume 'volume-8d910032-95a8-4e99-9b61-7f91786325a8' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:10:57.048700 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:10:57.182481 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:10:57.182669 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:10:57.186586 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3dc2b630-ad99-4fae-a4e9-811af8702b12) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8d910032-95a8-4e99-9b61-7f91786325a8', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:57.191047 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8c092f89-ea2b-4ea7-951e-5ca3c31a8057) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:10:57.220485 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Volume volume-8d910032-95a8-4e99-9b61-7f91786325a8 (8d910032-95a8-4e99-9b61-7f91786325a8): created successfully Sep 17 21:10:57.224410 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8c092f89-ea2b-4ea7-951e-5ca3c31a8057) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:10:57.232345 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Flow 'volume_create_manager' (6affacc0-8aa4-4fa0-bbcc-4c6b40193a2f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:10:57.260246 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-451d8006-26ea-4365-9677-334bd17d40f5 tempest-VolumesNegativeTest-1272947138 None] Created volume successfully. Sep 17 21:11:01.035164 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:11:01.035861 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:01.128743 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:01.182267 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:11:22.407326 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=35,cinder:INSERT=4,cinder:UPDATE=17,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:22.409063 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=15,cinder:INSERT=2,cinder:UPDATE=11 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:24.884312 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:11:24.964261 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Flow 'volume_create_manager' (ab14d240-4c17-4850-b0c8-b9fb19894919) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:11:25.003110 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6af54637-af9c-4b50-8cd7-f7196242b6f8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:25.042268 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6af54637-af9c-4b50-8cd7-f7196242b6f8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:11:24Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-1683447241',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6fa7e14863254ab3964ac907c168d9e6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:11:25Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:11:25Z,use_quota=True,user_id='c544e8e6f7a741c985c92ed7437397cd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:25.049037 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1abf17fb-fb3d-44a5-99c2-6464a604ffae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:25.053604 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1abf17fb-fb3d-44a5-99c2-6464a604ffae) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:25.061762 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b9a99da2-127a-46c3-ae8b-05a3d0868c45) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:25.067578 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b9a99da2-127a-46c3-ae8b-05a3d0868c45) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2', 'volume_name': 'volume-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:25.085102 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b1892739-7751-4d33-ad96-4d25882ea6ad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:25.099380 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b1892739-7751-4d33-ad96-4d25882ea6ad) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:25.106152 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (97634d9c-9ed1-4a97-89d6-15be4f51a999) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:25.113118 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Volume 06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2', 'volume_size': 1} Sep 17 21:11:25.113858 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] creating volume 'volume-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:11:25.114814 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:25.277703 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:11:25.278571 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:11:25.287995 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (97634d9c-9ed1-4a97-89d6-15be4f51a999) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:25.309440 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (09768a29-2fd5-4efd-9899-59d2f0372f71) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:25.344835 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Volume volume-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2 (06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2): created successfully Sep 17 21:11:25.366860 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (09768a29-2fd5-4efd-9899-59d2f0372f71) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:25.384227 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Flow 'volume_create_manager' (ab14d240-4c17-4850-b0c8-b9fb19894919) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:11:25.403955 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e9e7c4d2-a8a8-40bf-bea9-b2e8c2fdb010 tempest-AttachVolumeNegativeTest-1020448628 None] Created volume successfully. Sep 17 21:11:26.864402 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-c1db15b6-dea2-4956-a5d0-fa57587d78cb req-3d1f9a93-ef28-44d4-8843-beeb226d66bd tempest-AttachVolumeNegativeTest-1020448628 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:11:28.590343 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-c1db15b6-dea2-4956-a5d0-fa57587d78cb req-3d1f9a93-ef28-44d4-8843-beeb226d66bd tempest-AttachVolumeNegativeTest-1020448628 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.725s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:11:28.594302 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-c1db15b6-dea2-4956-a5d0-fa57587d78cb req-3d1f9a93-ef28-44d4-8843-beeb226d66bd tempest-AttachVolumeNegativeTest-1020448628 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:11:28.672485 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-c1db15b6-dea2-4956-a5d0-fa57587d78cb req-3d1f9a93-ef28-44d4-8843-beeb226d66bd tempest-AttachVolumeNegativeTest-1020448628 None] Connection info returned from driver {'name': 'volumes/volume-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '55f97e5d-7e68-4414-a838-cbe642ad645d'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:11:28.839423 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-c1db15b6-dea2-4956-a5d0-fa57587d78cb req-3d1f9a93-ef28-44d4-8843-beeb226d66bd tempest-AttachVolumeNegativeTest-1020448628 None] attachment_update completed successfully. Sep 17 21:11:33.029134 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-b0ef5191-6da9-44ce-ae12-b059214342ee req-b49efe3b-7baf-48d1-b9ca-8f8db210a625 tempest-AttachVolumeNegativeTest-1020448628 None] Terminate volume connection completed successfully. Sep 17 21:11:33.029764 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-b0ef5191-6da9-44ce-ae12-b059214342ee req-b49efe3b-7baf-48d1-b9ca-8f8db210a625 tempest-AttachVolumeNegativeTest-1020448628 None] Deleting attachment 55f97e5d-7e68-4414-a838-cbe642ad645d. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:11:41.629565 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] Acquiring lock "cinder-8d910032-95a8-4e99-9b61-7f91786325a8-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:11:41.665580 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] Lock "cinder-8d910032-95a8-4e99-9b61-7f91786325a8-delete_volume" acquired by "delete_volume" :: waited 0.040s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:11:41.781617 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:41.931027 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:11:41.931482 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] Volume volume-8d910032-95a8-4e99-9b61-7f91786325a8 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:11:41.953891 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] deleting rbd volume volume-8d910032-95a8-4e99-9b61-7f91786325a8 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:11:42.590285 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] Created reservations ['e958a3a6-0658-43ef-aad1-caa3bd7f202e', '6d2d57e8-80a6-4823-a4a4-96ebf6fbafef', 'e6331cef-3e46-495c-a08e-f6f9660690c1', '1da41962-d1b2-46d0-9b96-d95f2fd2fb43'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:11:42.899389 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:43.081083 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:43.185096 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:11:43.196421 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] Deleted volume successfully. Sep 17 21:11:43.229524 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-944ae1e6-d2d2-4e09-b43e-56c07c29e9b1 tempest-VolumesNegativeTest-1272947138 None] Lock "cinder-8d910032-95a8-4e99-9b61-7f91786325a8-delete_volume" released by "delete_volume" :: held 1.562s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:11:47.157231 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] Acquiring lock "cinder-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:11:47.173821 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] Lock "cinder-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2-delete_volume" acquired by "delete_volume" :: waited 0.017s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:11:47.250254 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:47.360168 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:11:47.360856 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] Volume volume-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:11:47.370876 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] deleting rbd volume volume-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:11:47.854453 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] Created reservations ['1cba5b17-b205-48f9-ba5e-0defb1bf1270', 'fd5c9f94-5dcf-4215-8ae8-cf74cbcbb57f', '3805dac8-4aaf-437e-ab87-7e19b492c50e', '10a848c3-ee54-4dfd-bdb3-2571dd04e96b'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:11:47.958290 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:48.054052 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:48.175816 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:11:48.180141 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] Deleted volume successfully. Sep 17 21:11:48.211169 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-168f4c53-0d45-42c4-9574-c887b3e79586 tempest-AttachVolumeNegativeTest-1020448628 None] Lock "cinder-06e1e35e-a1c2-4ffd-b75a-88d143d3a4e2-delete_volume" released by "delete_volume" :: held 1.037s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:11:48.362951 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:11:48.411309 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Flow 'volume_create_manager' (5a6b08c2-680e-43e2-8561-6a17e1537c29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:11:48.439266 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b1e3ffb1-9581-4d8e-8cff-105e0646ca89) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:48.539868 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b1e3ffb1-9581-4d8e-8cff-105e0646ca89) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:11:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotListTestJSON-Volume-1634444607',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d756b995-fc70-4393-b1b2-07b001eb3c63,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='96f88828c3634d04977ccf21eb350e88',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:11:48Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:11:48Z,use_quota=True,user_id='058e5024f3fd4f00af564894a781cb0f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:48.547260 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (db0bf5d3-ecd6-4194-88f7-ea00a93becf6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:48.551303 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (db0bf5d3-ecd6-4194-88f7-ea00a93becf6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:48.571762 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (378481e1-9e55-4d7c-aa46-7b3e619ecbfa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:48.587186 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (378481e1-9e55-4d7c-aa46-7b3e619ecbfa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd756b995-fc70-4393-b1b2-07b001eb3c63', 'volume_name': 'volume-d756b995-fc70-4393-b1b2-07b001eb3c63', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:48.597623 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (111803c5-0897-44cb-97d2-7d58b0bfe9e8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:48.604040 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (111803c5-0897-44cb-97d2-7d58b0bfe9e8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:48.611119 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ff0c4d24-bf89-4e37-b3d4-5ac8c8396e38) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:48.612464 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Volume d756b995-fc70-4393-b1b2-07b001eb3c63: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d756b995-fc70-4393-b1b2-07b001eb3c63', 'volume_size': 1} Sep 17 21:11:48.613065 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] creating volume 'volume-d756b995-fc70-4393-b1b2-07b001eb3c63' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:11:48.614093 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:48.709843 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] Acquiring lock "cinder-f04b4403-7625-47e5-a148-a05fd9ddc8e4-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:11:48.722467 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] Lock "cinder-f04b4403-7625-47e5-a148-a05fd9ddc8e4-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:11:48.764279 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:11:48.765769 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:11:48.776604 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ff0c4d24-bf89-4e37-b3d4-5ac8c8396e38) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d756b995-fc70-4393-b1b2-07b001eb3c63', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:48.782389 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3e4d2172-518c-4853-87aa-3388a2466e4a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:11:48.824583 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Volume volume-d756b995-fc70-4393-b1b2-07b001eb3c63 (d756b995-fc70-4393-b1b2-07b001eb3c63): created successfully Sep 17 21:11:48.829134 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3e4d2172-518c-4853-87aa-3388a2466e4a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:11:48.835148 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Flow 'volume_create_manager' (5a6b08c2-680e-43e2-8561-6a17e1537c29) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:11:48.872891 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:48.978845 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:11:48.979400 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] Volume volume-f04b4403-7625-47e5-a148-a05fd9ddc8e4 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:11:49.003210 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] deleting rbd volume volume-f04b4403-7625-47e5-a148-a05fd9ddc8e4 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:11:49.003210 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3bd076f0-cc96-4d82-b32d-d9649517456f tempest-VolumesSnapshotListTestJSON-611467425 None] Created volume successfully. Sep 17 21:11:49.505115 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=64,cinder:UPDATE=28,cinder:INSERT=7,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:49.514640 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] Created reservations ['18e4d27d-388b-4e65-aa48-a39e9439f44f', '9d807a29-5413-45a5-8225-8248921d301e', '56de2b19-5271-434f-b3d9-29b6e7b4afc9', '7b19c296-4327-41e2-a13a-37c844c9709e'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:11:49.676651 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:49.785071 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:49.836827 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:11:49.847101 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] Deleted volume successfully. Sep 17 21:11:49.919349 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3362d06a-980a-4f65-8274-39661e6f13e4 tempest-AttachVolumeNegativeTest-1020448628 None] Lock "cinder-f04b4403-7625-47e5-a148-a05fd9ddc8e4-delete_volume" released by "delete_volume" :: held 1.178s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:11:49.991497 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-02e5215d-0263-48b3-aa65-2e231b5ca551 tempest-VolumesSnapshotListTestJSON-611467425 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:50.447364 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] Acquiring lock "cinder-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:11:50.510425 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] Lock "cinder-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2-delete_volume" acquired by "delete_volume" :: waited 0.064s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:11:50.531888 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=58,cinder:UPDATE=34,cinder:DELETE=3,cinder:INSERT=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:11:50.548289 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-02e5215d-0263-48b3-aa65-2e231b5ca551 tempest-VolumesSnapshotListTestJSON-611467425 None] Create snapshot completed successfully Sep 17 21:11:50.644522 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:50.759880 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:11:50.760515 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] Volume volume-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:11:50.789002 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] deleting rbd volume volume-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:11:51.369787 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e58a97a5-8487-4707-a667-742cd8fa5806 tempest-VolumesSnapshotListTestJSON-611467425 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:51.557954 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] Created reservations ['25a2c3b0-3f93-496b-aa36-d8267db2b136', 'e41c9583-f270-4911-8b21-9da247b21953', '9a5341c4-44a3-454f-93cb-1bf8de018238', '8ddeb6ac-bbc2-4a1a-b290-02d401b6ef58'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:11:51.695177 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:51.823888 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:51.907454 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:11:51.912866 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] Deleted volume successfully. Sep 17 21:11:51.931033 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-41dce828-a597-4b0a-b284-e4c233200d91 tempest-AttachVolumeNegativeTest-1020448628 None] Lock "cinder-d1c88125-bbdc-4e6f-99b5-cc2bbb27caf2-delete_volume" released by "delete_volume" :: held 1.420s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:11:52.549398 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e58a97a5-8487-4707-a667-742cd8fa5806 tempest-VolumesSnapshotListTestJSON-611467425 None] Create snapshot completed successfully Sep 17 21:11:53.835522 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-89b1cb6d-80e4-4e5e-9398-2ca8854b89d4 tempest-VolumesSnapshotListTestJSON-611467425 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:54.635249 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-89b1cb6d-80e4-4e5e-9398-2ca8854b89d4 tempest-VolumesSnapshotListTestJSON-611467425 None] Create snapshot completed successfully Sep 17 21:11:56.348724 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-26b66a55-ba16-401e-bcd9-04edb67de564 tempest-VolumesSnapshotListTestJSON-611467425 None] Acquiring lock "cinder-26baef84-84e8-4537-a670-3e5be60ce0cd-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:11:56.362999 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-26b66a55-ba16-401e-bcd9-04edb67de564 tempest-VolumesSnapshotListTestJSON-611467425 None] Lock "cinder-26baef84-84e8-4537-a670-3e5be60ce0cd-delete_snapshot" acquired by "delete_snapshot" :: waited 0.014s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:11:56.366949 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-26b66a55-ba16-401e-bcd9-04edb67de564 tempest-VolumesSnapshotListTestJSON-611467425 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:56.822976 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-26b66a55-ba16-401e-bcd9-04edb67de564 tempest-VolumesSnapshotListTestJSON-611467425 None] Created reservations ['20627581-f4d4-40c0-bdbd-2bab62348d8c', '3fec1afe-96f9-4620-8eae-1417d9d47e2a', 'b0db3633-f8b6-439c-aecb-e802e528082d', 'b703b7ab-b06d-456a-bddc-bb42901f165c'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:11:56.916488 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-26b66a55-ba16-401e-bcd9-04edb67de564 tempest-VolumesSnapshotListTestJSON-611467425 None] Delete snapshot completed successfully. Sep 17 21:11:56.922234 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-26b66a55-ba16-401e-bcd9-04edb67de564 tempest-VolumesSnapshotListTestJSON-611467425 None] Lock "cinder-26baef84-84e8-4537-a670-3e5be60ce0cd-delete_snapshot" released by "delete_snapshot" :: held 0.559s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:11:57.578922 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a65f06ef-a3c9-4e01-bf92-506c1fd040ab tempest-VolumesSnapshotListTestJSON-611467425 None] Acquiring lock "cinder-b704d9e2-94ef-4ff4-b0f7-02595cdcd412-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:11:57.611300 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a65f06ef-a3c9-4e01-bf92-506c1fd040ab tempest-VolumesSnapshotListTestJSON-611467425 None] Lock "cinder-b704d9e2-94ef-4ff4-b0f7-02595cdcd412-delete_snapshot" acquired by "delete_snapshot" :: waited 0.031s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:11:57.623959 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a65f06ef-a3c9-4e01-bf92-506c1fd040ab tempest-VolumesSnapshotListTestJSON-611467425 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:11:58.856946 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-a65f06ef-a3c9-4e01-bf92-506c1fd040ab tempest-VolumesSnapshotListTestJSON-611467425 None] Created reservations ['ea91f3a1-16fc-4a01-93bb-a48e5a320fd5', 'cb8e9cbf-95b7-4e69-9fd7-ec9b3a6bb878', 'a462f771-2791-40e8-b92e-ea92c423ece3', '6c147f5c-5ed6-4cec-9fee-4fe8d32bcead'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:11:58.956015 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a65f06ef-a3c9-4e01-bf92-506c1fd040ab tempest-VolumesSnapshotListTestJSON-611467425 None] Delete snapshot completed successfully. Sep 17 21:11:58.962940 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a65f06ef-a3c9-4e01-bf92-506c1fd040ab tempest-VolumesSnapshotListTestJSON-611467425 None] Lock "cinder-b704d9e2-94ef-4ff4-b0f7-02595cdcd412-delete_snapshot" released by "delete_snapshot" :: held 1.353s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:11:59.850489 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-05656eb2-f46a-4398-9d3f-32ef6a291f1d tempest-VolumesSnapshotListTestJSON-611467425 None] Acquiring lock "cinder-12dfa1d9-5835-4951-ae7d-3655f22690c3-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:11:59.878115 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-05656eb2-f46a-4398-9d3f-32ef6a291f1d tempest-VolumesSnapshotListTestJSON-611467425 None] Lock "cinder-12dfa1d9-5835-4951-ae7d-3655f22690c3-delete_snapshot" acquired by "delete_snapshot" :: waited 0.028s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:11:59.885033 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-05656eb2-f46a-4398-9d3f-32ef6a291f1d tempest-VolumesSnapshotListTestJSON-611467425 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:12:00.883929 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=65,cinder:DELETE=3,cinder:UPDATE=25,cinder:INSERT=7 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:00.932540 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-05656eb2-f46a-4398-9d3f-32ef6a291f1d tempest-VolumesSnapshotListTestJSON-611467425 None] Created reservations ['f9eb0b51-e746-4c7e-a07e-832e93a67ce3', 'c96bd168-77b0-4425-8203-c94f47fb1a57', '80c29c15-665f-4123-9475-169fae1e1fdd', 'c5ae4394-efba-459d-943f-e81a6070a0da'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:12:01.017007 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-05656eb2-f46a-4398-9d3f-32ef6a291f1d tempest-VolumesSnapshotListTestJSON-611467425 None] Delete snapshot completed successfully. Sep 17 21:12:01.024211 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-05656eb2-f46a-4398-9d3f-32ef6a291f1d tempest-VolumesSnapshotListTestJSON-611467425 None] Lock "cinder-12dfa1d9-5835-4951-ae7d-3655f22690c3-delete_snapshot" released by "delete_snapshot" :: held 1.145s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:12:01.031403 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:12:01.032528 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:12:01.103941 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:12:01.201799 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:12:02.209972 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] Acquiring lock "cinder-d756b995-fc70-4393-b1b2-07b001eb3c63-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:12:02.244131 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] Lock "cinder-d756b995-fc70-4393-b1b2-07b001eb3c63-delete_volume" acquired by "delete_volume" :: waited 0.031s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:12:02.365426 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:12:02.484467 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:12:02.485151 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] Volume volume-d756b995-fc70-4393-b1b2-07b001eb3c63 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:12:02.499059 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] deleting rbd volume volume-d756b995-fc70-4393-b1b2-07b001eb3c63 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:12:02.916865 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] Created reservations ['cd96873b-3d23-470a-b9ce-4b3db92d4c65', '4cd811b4-eca2-4ec4-80c1-0281b3d0fce2', 'e68f8eff-cc40-4162-b256-aafb9a2fca19', 'f03e0da8-3aed-4da2-81da-ad8b5de595cf'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:12:03.027691 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:12:03.117046 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:12:03.199381 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:12:03.204498 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] Deleted volume successfully. Sep 17 21:12:03.243611 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d34658bf-4945-4a13-a4c5-183e2856766b tempest-VolumesSnapshotListTestJSON-611467425 None] Lock "cinder-d756b995-fc70-4393-b1b2-07b001eb3c63-delete_volume" released by "delete_volume" :: held 1.002s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:12:22.521312 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=55,cinder:INSERT=11,cinder:UPDATE=30,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:32.514302 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=20,cinder:UPDATE=18,cinder:INSERT=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:12:59.942436 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6f35142b-4f3e-445e-b437-2bdda629ad3d tempest-BackendsCapabilitiesAdminTestsJSON-2033100547 None] Obtained capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.3.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:b7523680-622f-42b1-bdc5-bda0e7b4a981:cinder:volumes', 'backend_state': 'up', 'qos_support': True, 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=133409) get_capabilities /opt/stack/cinder/cinder/volume/manager.py:4776}} Sep 17 21:12:59.987235 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-51f2493d-75d7-4373-a9db-5ce95f389742 tempest-BackendsCapabilitiesAdminTestsJSON-2033100547 None] Obtained capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.3.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:b7523680-622f-42b1-bdc5-bda0e7b4a981:cinder:volumes', 'backend_state': 'up', 'qos_support': True, 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=133409) get_capabilities /opt/stack/cinder/cinder/volume/manager.py:4776}} Sep 17 21:13:01.027798 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:13:01.029141 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:01.094158 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:01.183917 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:13:02.507699 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:07.714660 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-91f0e96e-83d5-4a70-89cc-3cb29214f5f3 tempest-GroupsTest-1245029510 None] Group tempest-Group1-389089923: creating Sep 17 21:13:07.730378 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:07.757185 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-91f0e96e-83d5-4a70-89cc-3cb29214f5f3 tempest-GroupsTest-1245029510 None] Group tempest-Group1-389089923: created successfully Sep 17 21:13:07.769087 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-91f0e96e-83d5-4a70-89cc-3cb29214f5f3 tempest-GroupsTest-1245029510 None] Create group completed successfully. Sep 17 21:13:08.991809 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6e5c1e7b-c369-43cd-a236-e60165d8072c tempest-GroupsTest-1245029510 None] Group tempest-Group2-1400363737: creating Sep 17 21:13:09.014529 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6e5c1e7b-c369-43cd-a236-e60165d8072c tempest-GroupsTest-1245029510 None] Group tempest-Group2-1400363737: created successfully Sep 17 21:13:09.026364 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6e5c1e7b-c369-43cd-a236-e60165d8072c tempest-GroupsTest-1245029510 None] Create group completed successfully. Sep 17 21:13:10.564386 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:13:10.598319 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Flow 'volume_create_manager' (4d907c32-b59f-4b1e-a43c-8224a5db0ae6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:13:10.604577 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (613a373a-0390-4ec8-8e15-831670d1997a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:10.624372 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (613a373a-0390-4ec8-8e15-831670d1997a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:13:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-volume-1606624974',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=5d437917-8d18-4d22-8ac6-3ee3fd2550ba,host='ncdv-master-1@ceph#ceph',id=db8fec26-7111-4c86-8da7-b02fc3278890,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9dd370c8e76749f69f6075678713bd39',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:13:10Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:13:10Z,use_quota=True,user_id='a61d2ca32f8c48cdb67b10fdced8a721',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(a4513844-75b3-4fbf-9026-7d11a17b3732),volume_type_id=a4513844-75b3-4fbf-9026-7d11a17b3732)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:10.630931 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (395fc19a-f6ff-429b-9c70-03aed952e302) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:10.636726 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (395fc19a-f6ff-429b-9c70-03aed952e302) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:10.645650 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8de765f2-aa4a-4d7f-829b-29a951c7efb5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:10.653166 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8de765f2-aa4a-4d7f-829b-29a951c7efb5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'db8fec26-7111-4c86-8da7-b02fc3278890', 'volume_name': 'volume-db8fec26-7111-4c86-8da7-b02fc3278890', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:10.661809 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4484eb81-53fb-4a32-8282-586c7159d528) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:10.666255 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4484eb81-53fb-4a32-8282-586c7159d528) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:10.672121 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ffb4cbf5-1ca9-4d4d-8fae-997ce7728ad4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:10.673636 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Volume db8fec26-7111-4c86-8da7-b02fc3278890: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-db8fec26-7111-4c86-8da7-b02fc3278890', 'volume_size': 1} Sep 17 21:13:10.674277 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] creating volume 'volume-db8fec26-7111-4c86-8da7-b02fc3278890' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:13:10.675858 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:10.771660 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:13:10.772588 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:13:10.777175 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ffb4cbf5-1ca9-4d4d-8fae-997ce7728ad4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-db8fec26-7111-4c86-8da7-b02fc3278890', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:10.781971 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bf59da8d-e06f-48e2-a0f4-a29209c24e59) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:10.797859 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Volume volume-db8fec26-7111-4c86-8da7-b02fc3278890 (db8fec26-7111-4c86-8da7-b02fc3278890): created successfully Sep 17 21:13:10.802299 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bf59da8d-e06f-48e2-a0f4-a29209c24e59) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:10.808716 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Flow 'volume_create_manager' (4d907c32-b59f-4b1e-a43c-8224a5db0ae6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:13:10.821631 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c772a9f0-1995-4c2d-853f-21c5b5a7eda0 tempest-GroupsTest-1245029510 None] Created volume successfully. Sep 17 21:13:12.509971 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:12.585639 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:13:12.586062 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] Volume volume-db8fec26-7111-4c86-8da7-b02fc3278890 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:13:12.594008 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] deleting rbd volume volume-db8fec26-7111-4c86-8da7-b02fc3278890 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:13:12.862145 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] Created reservations ['1ce8fb63-628f-4ed1-aec5-dd99effa0932'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:12.886268 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-2048805278 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-2048805278, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:13:12.888708 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-2048805278 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-2048805278, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:13:12.906127 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] Created reservations ['6549d403-e2c4-41e6-8ab6-6d8df28b16d2', '7205340a-01c2-4cf8-8ef1-424960f20e39', 'ca5c3b42-e82f-4723-93b5-4a6e0d57a4ce', 'ed80b54f-8687-4ccd-82a6-c70f9abecd3c'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:13.002888 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:13.054197 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:13.120594 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:13:13.126809 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-018bef7e-a38f-4528-a759-f07586de369d tempest-GroupsTest-1245029510 None] Delete group completed successfully. Sep 17 21:13:13.888137 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-f08ccaa9-8c82-4d69-900a-131197045af2 tempest-GroupsTest-1245029510 None] Created reservations ['a2a91477-7405-4d05-8cb3-2c22cecc76b5'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:13.962282 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f08ccaa9-8c82-4d69-900a-131197045af2 tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:14.027560 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f08ccaa9-8c82-4d69-900a-131197045af2 tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:14.102128 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-f08ccaa9-8c82-4d69-900a-131197045af2 tempest-GroupsTest-1245029510 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:13:14.111405 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f08ccaa9-8c82-4d69-900a-131197045af2 tempest-GroupsTest-1245029510 None] Delete group completed successfully. Sep 17 21:13:15.714366 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-9622e25d-d55a-470d-a252-eb78b000d1cf tempest-GroupsTest-1245029510 None] Group tempest-GroupsTest-Group-751964920: creating Sep 17 21:13:15.753694 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-9622e25d-d55a-470d-a252-eb78b000d1cf tempest-GroupsTest-1245029510 None] Group tempest-GroupsTest-Group-751964920: created successfully Sep 17 21:13:15.766993 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-9622e25d-d55a-470d-a252-eb78b000d1cf tempest-GroupsTest-1245029510 None] Create group completed successfully. Sep 17 21:13:17.596833 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:13:17.634572 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Flow 'volume_create_manager' (9e657319-d68c-42ec-8546-5fc54774bf5c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:13:17.659491 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (44acc240-5008-4fc0-bf64-2e6d5132e5fb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:17.699159 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (44acc240-5008-4fc0-bf64-2e6d5132e5fb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:13:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsTest-Volume-515420800',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=329c0c30-a1f2-4f65-95de-46195ba14508,host='ncdv-master-1@ceph#ceph',id=fdae68f2-cf75-4d36-b99e-039509d00a76,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9dd370c8e76749f69f6075678713bd39',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:13:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:13:18Z,use_quota=True,user_id='a61d2ca32f8c48cdb67b10fdced8a721',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(72a1799b-afff-4967-8963-f5859c7cb476),volume_type_id=72a1799b-afff-4967-8963-f5859c7cb476)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:17.706612 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (877f0912-6678-4392-9523-cff062fe28c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:17.712082 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (877f0912-6678-4392-9523-cff062fe28c7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:17.717797 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (900e9cc3-ecd2-4038-b923-037d1b0838bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:17.741749 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (900e9cc3-ecd2-4038-b923-037d1b0838bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'fdae68f2-cf75-4d36-b99e-039509d00a76', 'volume_name': 'volume-fdae68f2-cf75-4d36-b99e-039509d00a76', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:17.751023 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4fb2127b-97c7-4e56-9753-b39b832ecb80) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:17.755661 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4fb2127b-97c7-4e56-9753-b39b832ecb80) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:17.762294 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2f25d115-3bec-4a20-be46-dc9e13b61b7e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:17.763525 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Volume fdae68f2-cf75-4d36-b99e-039509d00a76: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-fdae68f2-cf75-4d36-b99e-039509d00a76', 'volume_size': 1} Sep 17 21:13:17.764138 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] creating volume 'volume-fdae68f2-cf75-4d36-b99e-039509d00a76' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:13:17.765382 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:17.893969 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:13:17.895174 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:13:17.901565 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2f25d115-3bec-4a20-be46-dc9e13b61b7e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-fdae68f2-cf75-4d36-b99e-039509d00a76', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:17.910888 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3898faac-1531-40c1-be70-88571a3c7ae2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:17.926853 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Volume volume-fdae68f2-cf75-4d36-b99e-039509d00a76 (fdae68f2-cf75-4d36-b99e-039509d00a76): created successfully Sep 17 21:13:17.932283 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3898faac-1531-40c1-be70-88571a3c7ae2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:17.943113 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Flow 'volume_create_manager' (9e657319-d68c-42ec-8546-5fc54774bf5c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:13:17.965212 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-617ed14b-95c1-4725-8a67-e7ded75cc7c7 tempest-GroupsTest-1245029510 None] Created volume successfully. Sep 17 21:13:19.249036 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:13:19.298478 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Flow 'volume_create_manager' (871b0525-08e4-4179-9d05-58831c38d042) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:13:19.309106 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1a6e1b9a-6874-4a65-a3a0-ed59b0a16c55) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:19.337865 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1a6e1b9a-6874-4a65-a3a0-ed59b0a16c55) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:13:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsTest-Volume-1191700338',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=329c0c30-a1f2-4f65-95de-46195ba14508,host='ncdv-master-1@ceph#ceph',id=18d90f66-ec19-4c6e-9c5f-465ba42c4f33,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9dd370c8e76749f69f6075678713bd39',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:13:19Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:13:19Z,use_quota=True,user_id='a61d2ca32f8c48cdb67b10fdced8a721',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(72a1799b-afff-4967-8963-f5859c7cb476),volume_type_id=72a1799b-afff-4967-8963-f5859c7cb476)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:19.345032 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fd2f60c0-0297-43b8-8fd3-9d78b02a0312) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:19.350895 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fd2f60c0-0297-43b8-8fd3-9d78b02a0312) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:19.357145 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1dcd3137-fb31-47dd-9058-67df87caf604) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:19.366200 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1dcd3137-fb31-47dd-9058-67df87caf604) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '18d90f66-ec19-4c6e-9c5f-465ba42c4f33', 'volume_name': 'volume-18d90f66-ec19-4c6e-9c5f-465ba42c4f33', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:19.372059 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (98a2ba7d-0496-4784-bd94-1b680593fed4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:19.377452 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (98a2ba7d-0496-4784-bd94-1b680593fed4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:19.382521 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (367c3932-061f-4daf-9c89-deb4c25bb725) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:19.383827 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Volume 18d90f66-ec19-4c6e-9c5f-465ba42c4f33: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-18d90f66-ec19-4c6e-9c5f-465ba42c4f33', 'volume_size': 1} Sep 17 21:13:19.384362 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] creating volume 'volume-18d90f66-ec19-4c6e-9c5f-465ba42c4f33' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:13:19.385151 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:19.496031 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:13:19.496855 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:13:19.506439 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (367c3932-061f-4daf-9c89-deb4c25bb725) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-18d90f66-ec19-4c6e-9c5f-465ba42c4f33', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:19.515393 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a7c97ef7-1a10-4c44-b7ea-c798df15d6bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:19.529049 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Volume volume-18d90f66-ec19-4c6e-9c5f-465ba42c4f33 (18d90f66-ec19-4c6e-9c5f-465ba42c4f33): created successfully Sep 17 21:13:19.533955 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a7c97ef7-1a10-4c44-b7ea-c798df15d6bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:19.540722 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Flow 'volume_create_manager' (871b0525-08e4-4179-9d05-58831c38d042) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:13:19.564937 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5b3eef95-020f-46bc-99ea-6b5192ceb1c5 tempest-GroupsTest-1245029510 None] Created volume successfully. Sep 17 21:13:20.528658 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8d8ea9ee-7a79-4ad5-ab51-24dfce6aa09c tempest-GroupsTest-1245029510 None] Update group completed successfully. Sep 17 21:13:21.871816 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e16b46db-0849-4f12-a7ab-bc127ed9cd88 tempest-GroupsTest-1245029510 None] Update group completed successfully. Sep 17 21:13:23.523832 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:23.613748 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:13:23.614484 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] Volume volume-18d90f66-ec19-4c6e-9c5f-465ba42c4f33 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:13:23.626549 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] deleting rbd volume volume-18d90f66-ec19-4c6e-9c5f-465ba42c4f33 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:13:23.915658 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:24.012983 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:13:24.014895 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] Volume volume-fdae68f2-cf75-4d36-b99e-039509d00a76 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:13:24.027825 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] deleting rbd volume volume-fdae68f2-cf75-4d36-b99e-039509d00a76 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:13:24.384804 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=66,cinder:UPDATE=30,cinder:INSERT=2,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:24.404657 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=75,cinder:UPDATE=20,cinder:DELETE=1,cinder:INSERT=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:24.438880 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] Created reservations ['738e05cd-2744-4051-b384-bbdc4c25a3bb'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:24.470948 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-356558167 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-356558167, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:13:24.472419 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-356558167 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-356558167, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:13:24.496925 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] Created reservations ['7b1f3820-8bc4-48fd-bb70-dbe35667b5e0', '893309d7-6be9-4ec1-b2f1-8374062944b3', '14681106-639b-48e4-8b53-d2db75db108d', 'ef33e15d-09a8-4823-af17-233ab600ba68'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:24.648506 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] Created reservations ['fc273748-237d-4cbb-8346-d4e5e75ec5d2', '70cc531e-74f4-4223-8b0e-2b17c1dcc890', '5618cb9b-0c67-407e-8e4c-38d7847456f1', 'ae043583-7667-456f-8fc2-4c135f2f1070'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:24.790192 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:24.843641 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:24.922541 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:13:24.927023 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-29dbc686-621f-4210-b299-c7afa902c7a5 tempest-GroupsTest-1245029510 None] Delete group completed successfully. Sep 17 21:13:32.883762 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:13:32.923939 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Flow 'volume_create_manager' (e4990110-649e-49eb-8d3a-91ee0f11bb67) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:13:32.932019 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e7d13f3f-d9af-43fa-801d-633894ebcc7e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:32.953506 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e7d13f3f-d9af-43fa-801d-633894ebcc7e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:13:33Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotsActionsTest-Volume-1185082819',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='203653c6a42345ee8fe68f0a0bc64a7d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:13:33Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:13:33Z,use_quota=True,user_id='5ae3425f3c704ece85cf9c6b54949fac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:32.981333 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3284239c-6214-4628-93c2-8cc7728b9144) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:32.987484 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3284239c-6214-4628-93c2-8cc7728b9144) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:32.994025 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (18ed83d2-2a71-48fb-ada1-6bb7c1e7efde) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:32.999713 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (18ed83d2-2a71-48fb-ada1-6bb7c1e7efde) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab', 'volume_name': 'volume-dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:33.005759 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e5d7a850-4ea1-471b-b98b-6f99c6ad8173) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:33.010417 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e5d7a850-4ea1-471b-b98b-6f99c6ad8173) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:33.015699 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c8e513ca-8953-4107-b321-f4f338966fef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:33.017033 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Volume dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab', 'volume_size': 1} Sep 17 21:13:33.017636 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] creating volume 'volume-dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:13:33.018680 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:33.158306 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:13:33.158680 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:13:33.162582 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c8e513ca-8953-4107-b321-f4f338966fef) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:33.173142 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (be85d3b1-8364-4eb8-97a1-ace4ae9d5e9a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:13:33.195589 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Volume volume-dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab (dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab): created successfully Sep 17 21:13:33.201063 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (be85d3b1-8364-4eb8-97a1-ace4ae9d5e9a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:13:33.209188 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Flow 'volume_create_manager' (e4990110-649e-49eb-8d3a-91ee0f11bb67) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:13:33.224726 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-65d7be22-372f-4ef3-b89f-f4b6ce659930 tempest-SnapshotsActionsTest-1964124327 None] Created volume successfully. Sep 17 21:13:34.405102 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ac9c93e5-ce60-42b8-9045-fb56320daaa7 tempest-SnapshotsActionsTest-1964124327 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:35.622730 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ac9c93e5-ce60-42b8-9045-fb56320daaa7 tempest-SnapshotsActionsTest-1964124327 None] Create snapshot completed successfully Sep 17 21:13:37.542484 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a6c39870-f4d8-4e3f-a762-3fb2ded38b15 tempest-SnapshotsActionsTest-1964124327 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:38.701784 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a6c39870-f4d8-4e3f-a762-3fb2ded38b15 tempest-SnapshotsActionsTest-1964124327 None] Create snapshot completed successfully Sep 17 21:13:39.865331 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ef8b992a-29e8-4096-9222-90d6f6afe6f0 tempest-SnapshotsActionsTest-545854842 None] Acquiring lock "cinder-8bf6ffee-3c39-4e8d-ba6e-944f2a854473-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:13:39.892907 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ef8b992a-29e8-4096-9222-90d6f6afe6f0 tempest-SnapshotsActionsTest-545854842 None] Lock "cinder-8bf6ffee-3c39-4e8d-ba6e-944f2a854473-delete_snapshot" acquired by "delete_snapshot" :: waited 0.028s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:13:39.897929 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ef8b992a-29e8-4096-9222-90d6f6afe6f0 tempest-SnapshotsActionsTest-545854842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:40.732375 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-ef8b992a-29e8-4096-9222-90d6f6afe6f0 tempest-SnapshotsActionsTest-545854842 None] Created reservations ['50fc9786-bb1f-46c3-ac26-2170313ab0f1', '857a2433-a3da-4590-abe5-ad694a82f4c3', 'a65b159d-a9bb-4c88-bbd1-099ec24a6a56', '44fcdf14-965f-4667-b4fa-7d7b43b7de1c'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:40.786903 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ef8b992a-29e8-4096-9222-90d6f6afe6f0 tempest-SnapshotsActionsTest-545854842 None] Delete snapshot completed successfully. Sep 17 21:13:40.795941 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ef8b992a-29e8-4096-9222-90d6f6afe6f0 tempest-SnapshotsActionsTest-545854842 None] Lock "cinder-8bf6ffee-3c39-4e8d-ba6e-944f2a854473-delete_snapshot" released by "delete_snapshot" :: held 0.903s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:13:41.469806 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ccefe3d2-b933-4a09-b84c-1b53d9a62244 tempest-SnapshotsActionsTest-1964124327 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:41.765394 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ccefe3d2-b933-4a09-b84c-1b53d9a62244 tempest-SnapshotsActionsTest-1964124327 None] Create snapshot completed successfully Sep 17 21:13:42.826938 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f24a0a72-734a-4a34-b25c-f867f79048d1 tempest-SnapshotsActionsTest-545854842 None] Acquiring lock "cinder-ae018edf-61a0-49a9-b889-e0a75ce7be99-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:13:42.848510 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f24a0a72-734a-4a34-b25c-f867f79048d1 tempest-SnapshotsActionsTest-545854842 None] Lock "cinder-ae018edf-61a0-49a9-b889-e0a75ce7be99-delete_snapshot" acquired by "delete_snapshot" :: waited 0.022s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:13:42.853135 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f24a0a72-734a-4a34-b25c-f867f79048d1 tempest-SnapshotsActionsTest-545854842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:43.903528 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=65,cinder:INSERT=7,cinder:UPDATE=26,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:43.943941 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-f24a0a72-734a-4a34-b25c-f867f79048d1 tempest-SnapshotsActionsTest-545854842 None] Created reservations ['41591dba-2bdb-47ac-a5e0-79e77ba4a275', '2e45e404-2d8c-4ec7-8381-16473e3fc325', '0217bd25-3b25-43e5-962e-acd2aa6bbe6b', '0cd9be94-c13a-4be9-8b9c-ad745bcb50a7'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:44.011582 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=51,cinder:INSERT=10,cinder:UPDATE=37,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:13:44.026616 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f24a0a72-734a-4a34-b25c-f867f79048d1 tempest-SnapshotsActionsTest-545854842 None] Delete snapshot completed successfully. Sep 17 21:13:44.043179 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f24a0a72-734a-4a34-b25c-f867f79048d1 tempest-SnapshotsActionsTest-545854842 None] Lock "cinder-ae018edf-61a0-49a9-b889-e0a75ce7be99-delete_snapshot" released by "delete_snapshot" :: held 1.195s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:13:45.618828 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-49fef514-2cd3-4f99-81b7-77529948849a tempest-SnapshotsActionsTest-1964124327 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:46.918945 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-49fef514-2cd3-4f99-81b7-77529948849a tempest-SnapshotsActionsTest-1964124327 None] Create snapshot completed successfully Sep 17 21:13:48.025940 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-e9bd18f7-e979-4775-a1ee-0aa5262689a4 tempest-SnapshotsActionsTest-545854842 None] Acquiring lock "cinder-ef5213e8-a9a6-418c-81e9-fba7a8349567-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:13:48.051459 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-e9bd18f7-e979-4775-a1ee-0aa5262689a4 tempest-SnapshotsActionsTest-545854842 None] Lock "cinder-ef5213e8-a9a6-418c-81e9-fba7a8349567-delete_snapshot" acquired by "delete_snapshot" :: waited 0.028s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:13:48.058030 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9bd18f7-e979-4775-a1ee-0aa5262689a4 tempest-SnapshotsActionsTest-545854842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:49.036815 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-e9bd18f7-e979-4775-a1ee-0aa5262689a4 tempest-SnapshotsActionsTest-545854842 None] Created reservations ['3eca803e-bb5a-42cf-b2f6-4fdf11d20d94', 'b90981f5-d412-4bed-a242-e49e8dd56048', '9db1a2b4-1280-4ed7-acaa-0808dd09dcf0', '26ad0014-a397-4c0d-9152-f233735e7186'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:49.119115 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e9bd18f7-e979-4775-a1ee-0aa5262689a4 tempest-SnapshotsActionsTest-545854842 None] Delete snapshot completed successfully. Sep 17 21:13:49.132482 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-e9bd18f7-e979-4775-a1ee-0aa5262689a4 tempest-SnapshotsActionsTest-545854842 None] Lock "cinder-ef5213e8-a9a6-418c-81e9-fba7a8349567-delete_snapshot" released by "delete_snapshot" :: held 1.079s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:13:49.765219 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8993049f-6084-4403-ab5c-0bab4ed3500e tempest-SnapshotsActionsTest-1964124327 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:51.011879 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8993049f-6084-4403-ab5c-0bab4ed3500e tempest-SnapshotsActionsTest-1964124327 None] Create snapshot completed successfully Sep 17 21:13:52.091097 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5f8fed92-1617-40d1-8aff-60c88c067e8c tempest-SnapshotsActionsTest-545854842 None] Acquiring lock "cinder-e99dfc81-ce1b-45b3-99be-982fbfdc443d-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:13:52.114239 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5f8fed92-1617-40d1-8aff-60c88c067e8c tempest-SnapshotsActionsTest-545854842 None] Lock "cinder-e99dfc81-ce1b-45b3-99be-982fbfdc443d-delete_snapshot" acquired by "delete_snapshot" :: waited 0.023s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:13:52.116074 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5f8fed92-1617-40d1-8aff-60c88c067e8c tempest-SnapshotsActionsTest-545854842 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:53.150016 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-5f8fed92-1617-40d1-8aff-60c88c067e8c tempest-SnapshotsActionsTest-545854842 None] Created reservations ['d1cb6aec-6cf6-42bb-92dd-935dcbe4e3ff', 'df41b57f-c87d-4c12-9a01-ab270d289660', '7196e692-a52e-4c3a-8697-48dc2238e7a1', '805efa5d-fcc7-445e-b6ce-b51c49cf98ae'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:53.270814 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5f8fed92-1617-40d1-8aff-60c88c067e8c tempest-SnapshotsActionsTest-545854842 None] Delete snapshot completed successfully. Sep 17 21:13:53.280207 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5f8fed92-1617-40d1-8aff-60c88c067e8c tempest-SnapshotsActionsTest-545854842 None] Lock "cinder-e99dfc81-ce1b-45b3-99be-982fbfdc443d-delete_snapshot" released by "delete_snapshot" :: held 1.166s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:13:55.248627 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f215c8e8-091e-48e3-b615-efe65a3b398b tempest-SnapshotsActionsTest-1964124327 None] Acquiring lock "cinder-0e8cdaa3-27fb-429b-9cdb-52d816b705f8-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:13:55.295923 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f215c8e8-091e-48e3-b615-efe65a3b398b tempest-SnapshotsActionsTest-1964124327 None] Lock "cinder-0e8cdaa3-27fb-429b-9cdb-52d816b705f8-delete_snapshot" acquired by "delete_snapshot" :: waited 0.047s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:13:55.298250 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f215c8e8-091e-48e3-b615-efe65a3b398b tempest-SnapshotsActionsTest-1964124327 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:56.629779 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-f215c8e8-091e-48e3-b615-efe65a3b398b tempest-SnapshotsActionsTest-1964124327 None] Created reservations ['0dbbb551-8b8d-49d8-9725-7dedff3bae7f', 'f5c0afc5-c1ab-46bd-ae2c-ac6b6af26a62', 'b2c5fffd-7327-4480-b2ac-1716558fbc3a', '96b4472b-2c74-48b1-b853-a2ae0e0faf24'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:56.697780 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f215c8e8-091e-48e3-b615-efe65a3b398b tempest-SnapshotsActionsTest-1964124327 None] Delete snapshot completed successfully. Sep 17 21:13:56.703782 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f215c8e8-091e-48e3-b615-efe65a3b398b tempest-SnapshotsActionsTest-1964124327 None] Lock "cinder-0e8cdaa3-27fb-429b-9cdb-52d816b705f8-delete_snapshot" released by "delete_snapshot" :: held 1.408s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:13:57.491702 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] Acquiring lock "cinder-dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:13:57.503524 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] Lock "cinder-dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:13:57.584666 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:57.681656 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:13:57.683766 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] Volume volume-dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:13:57.699743 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] deleting rbd volume volume-dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:13:58.131817 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] Created reservations ['6bb294eb-9615-4185-9d6d-52d38e232c56', '5e0f9a80-d4ee-4dc1-a951-bc417faf6700', '1ba6d39e-6b3e-4888-a6bd-36359d551841', '09890275-045c-487e-84d0-08878ca6c046'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:13:58.214817 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:58.287781 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:13:58.344836 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:13:58.351361 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] Deleted volume successfully. Sep 17 21:13:58.370337 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2ea3280e-0c57-4ab5-8dde-4216a83be91b tempest-SnapshotsActionsTest-1964124327 None] Lock "cinder-dfb50bcb-6f32-4ea9-ac77-6fa8f0c719ab-delete_volume" released by "delete_volume" :: held 0.867s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:14:01.026345 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:14:01.027069 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:01.084511 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:01.169072 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:14:05.164901 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=55,cinder:UPDATE=36,cinder:DELETE=2,cinder:INSERT=7 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:05.196253 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-58325cd5-159b-4c61-913f-9ba7267a948f req-7c8d634b-0418-468e-8c12-48c8403523d7 tempest-TaggedBootDevicesTest_v242-970179712 None] Terminate volume connection completed successfully. Sep 17 21:14:05.205930 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-58325cd5-159b-4c61-913f-9ba7267a948f req-7c8d634b-0418-468e-8c12-48c8403523d7 tempest-TaggedBootDevicesTest_v242-970179712 None] Deleting attachment 1c7b252e-c0f1-4851-a2fb-a018a2d2b497. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:14:05.426309 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-58325cd5-159b-4c61-913f-9ba7267a948f req-6931fae1-48e3-4131-b42c-a2c53a6eb31a tempest-TaggedBootDevicesTest_v242-970179712 None] Terminate volume connection completed successfully. Sep 17 21:14:05.426777 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-58325cd5-159b-4c61-913f-9ba7267a948f req-6931fae1-48e3-4131-b42c-a2c53a6eb31a tempest-TaggedBootDevicesTest_v242-970179712 None] Deleting attachment 0e4663f0-c4da-45d4-abd5-12f2defcec88. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:14:05.675539 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-58325cd5-159b-4c61-913f-9ba7267a948f req-59518fc5-be03-4b91-a513-bb08a2c47e14 tempest-TaggedBootDevicesTest_v242-970179712 None] Terminate volume connection completed successfully. Sep 17 21:14:05.676225 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-58325cd5-159b-4c61-913f-9ba7267a948f req-59518fc5-be03-4b91-a513-bb08a2c47e14 tempest-TaggedBootDevicesTest_v242-970179712 None] Deleting attachment 693624fa-7fb5-4bdd-ab09-a5049c496985. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:14:15.648171 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:22.524421 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=63,cinder:INSERT=9,cinder:UPDATE=24,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:22.937866 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] Acquiring lock "cinder-63216c51-2591-4185-b165-a1abff51874d-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:14:22.948852 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] Lock "cinder-63216c51-2591-4185-b165-a1abff51874d-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:14:23.054452 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:23.163099 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:14:23.163512 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] Volume volume-63216c51-2591-4185-b165-a1abff51874d is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:14:23.184335 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] deleting rbd volume volume-63216c51-2591-4185-b165-a1abff51874d {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:14:23.662296 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] Created reservations ['0c2071d9-3815-419a-8ac5-516e2019d2c3', 'e1dc10bf-537f-4e60-986e-1bad2503657f', '456395cf-2b59-4707-9951-af79a378b11b', 'fb40798a-cec9-42e2-b515-aeaa71c2c21a'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:14:23.786734 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:23.856985 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:23.918288 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:14:23.927379 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] Deleted volume successfully. Sep 17 21:14:23.942792 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-28c13513-cfba-47b9-881e-e9d7768b4bd4 tempest-TaggedBootDevicesTest_v242-970179712 None] Lock "cinder-63216c51-2591-4185-b165-a1abff51874d-delete_volume" released by "delete_volume" :: held 0.994s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:14:24.130590 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] Acquiring lock "cinder-03debbe2-b045-44c5-a1ec-8136363e731b-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:14:24.144735 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] Lock "cinder-03debbe2-b045-44c5-a1ec-8136363e731b-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:14:24.216976 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:24.297305 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:14:24.297927 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] Volume volume-03debbe2-b045-44c5-a1ec-8136363e731b is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:14:24.303261 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] deleting rbd volume volume-03debbe2-b045-44c5-a1ec-8136363e731b {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:14:24.760427 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] Created reservations ['c6fbdedd-f6e4-445e-a865-097f36075d1b', '160cb49c-18d3-4acd-86c0-cc90e07e15f7', 'df29c478-521a-4b28-a8dd-8391009e676d', 'a0662e77-80fe-4747-bd70-f3b02c056df3'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:14:24.862392 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:24.924228 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:24.998672 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:14:25.010224 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] Deleted volume successfully. Sep 17 21:14:25.038939 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5cbd268c-f51a-45ef-ab94-16c41e4643f9 tempest-TaggedBootDevicesTest_v242-970179712 None] Lock "cinder-03debbe2-b045-44c5-a1ec-8136363e731b-delete_volume" released by "delete_volume" :: held 0.895s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:14:25.414077 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] Acquiring lock "cinder-c9e0dd22-ca21-4935-99e2-ec55ffcac017-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:14:25.441285 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] Lock "cinder-c9e0dd22-ca21-4935-99e2-ec55ffcac017-delete_volume" acquired by "delete_volume" :: waited 0.027s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:14:25.538504 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:25.668671 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:14:25.700351 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] deleting rbd volume volume-c9e0dd22-ca21-4935-99e2-ec55ffcac017 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:14:26.255761 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:14:26.294878 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] deleting parent snapshot volume-c9e0dd22-ca21-4935-99e2-ec55ffcac017.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:14:27.065860 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] Created reservations ['3c8ef129-39e5-4fe0-9c39-5ba625923645', '82f8b43f-9da9-4430-ac4c-f178d7836fd5', '9570a696-db23-4dea-9c96-69b83381e767', '4141828d-77ba-48ea-926b-ff58e2ec1283'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:14:27.186309 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:27.255190 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:27.326025 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:14:27.336084 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] Deleted volume successfully. Sep 17 21:14:27.357624 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-897925bb-9ef6-4e6f-92da-6994b464d23c tempest-TaggedBootDevicesTest_v242-970179712 None] Lock "cinder-c9e0dd22-ca21-4935-99e2-ec55ffcac017-delete_volume" released by "delete_volume" :: held 1.916s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:14:28.968986 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:14:29.003350 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Flow 'volume_create_manager' (8c6ca74b-7944-43f3-b245-5597eb0650da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:14:29.010644 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (53caf6a8-1260-44f0-a189-1609c6057fec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:29.030918 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (53caf6a8-1260-44f0-a189-1609c6057fec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:14:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Volume-1863764906',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4adec8bd-67e4-4d63-8177-46e0243e55c6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='7e0169496f8740ca83bc6f92594ca95d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:14:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:14:29Z,use_quota=True,user_id='0042863b26ca408a9ab608edba9951ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:29.038293 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (877ef501-06d7-4383-a353-3b9e66f2644a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:29.043539 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (877ef501-06d7-4383-a353-3b9e66f2644a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:29.049029 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff0d9e4d-9d97-4897-bf96-c8b3d5be91d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:29.054831 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff0d9e4d-9d97-4897-bf96-c8b3d5be91d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4adec8bd-67e4-4d63-8177-46e0243e55c6', 'volume_name': 'volume-4adec8bd-67e4-4d63-8177-46e0243e55c6', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:29.066650 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a76044ff-b07f-4de9-9bd4-13767df45b9e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:29.071688 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a76044ff-b07f-4de9-9bd4-13767df45b9e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:29.077496 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (139e6d16-2d58-432f-8d3d-b502d979ebcf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:29.078770 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Volume 4adec8bd-67e4-4d63-8177-46e0243e55c6: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4adec8bd-67e4-4d63-8177-46e0243e55c6', 'volume_size': 1} Sep 17 21:14:29.079402 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] creating volume 'volume-4adec8bd-67e4-4d63-8177-46e0243e55c6' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:14:29.080417 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:29.185231 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:14:29.185643 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:14:29.192739 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (139e6d16-2d58-432f-8d3d-b502d979ebcf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4adec8bd-67e4-4d63-8177-46e0243e55c6', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:29.197562 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f1d44c5f-ac87-419b-9b16-bf0b63d4bd81) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:29.210882 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Volume volume-4adec8bd-67e4-4d63-8177-46e0243e55c6 (4adec8bd-67e4-4d63-8177-46e0243e55c6): created successfully Sep 17 21:14:29.216842 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f1d44c5f-ac87-419b-9b16-bf0b63d4bd81) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:29.223340 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Flow 'volume_create_manager' (8c6ca74b-7944-43f3-b245-5597eb0650da) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:14:29.241956 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d75ee08e-1157-412d-a0e2-ba60e7197cec tempest-VolumesBackupsAdminTest-841724568 None] Created volume successfully. Sep 17 21:14:30.287394 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-590be1df-4c64-4814-8071-71240aa10bf3 tempest-VolumesBackupsAdminTest-841724568 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-17T21:14:30Z,data_timestamp=2022-09-17T21:14:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-1395004441',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=c91269ce-6685-486d-80a1-f85db4d9cc4e,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='7e0169496f8740ca83bc6f92594ca95d',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='0042863b26ca408a9ab608edba9951ac',volume_id=4adec8bd-67e4-4d63-8177-46e0243e55c6) Sep 17 21:14:30.289687 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.backup.rpcapi [None req-590be1df-4c64-4814-8071-71240aa10bf3 tempest-VolumesBackupsAdminTest-841724568 None] continue_backup in rpcapi backup_id c91269ce-6685-486d-80a1-f85db4d9cc4e {{(pid=133409) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 17 21:14:30.505731 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-590be1df-4c64-4814-8071-71240aa10bf3 tempest-VolumesBackupsAdminTest-841724568 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:14:32.120243 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-590be1df-4c64-4814-8071-71240aa10bf3 tempest-VolumesBackupsAdminTest-841724568 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.614s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:14:32.121239 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-590be1df-4c64-4814-8071-71240aa10bf3 tempest-VolumesBackupsAdminTest-841724568 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-4adec8bd-67e4-4d63-8177-46e0243e55c6', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '4adec8bd-67e4-4d63-8177-46e0243e55c6', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:14:32.148290 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-590be1df-4c64-4814-8071-71240aa10bf3 tempest-VolumesBackupsAdminTest-841724568 None] Initialize volume connection completed successfully. Sep 17 21:14:34.219148 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-590be1df-4c64-4814-8071-71240aa10bf3 tempest-VolumesBackupsAdminTest-841724568 None] Terminate volume connection completed successfully. Sep 17 21:14:34.254983 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-590be1df-4c64-4814-8071-71240aa10bf3 tempest-VolumesBackupsAdminTest-841724568 None] Remove volume export completed successfully. Sep 17 21:14:36.883585 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:14:36.932315 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Flow 'volume_create_manager' (570b2fc6-1f86-4c26-9ae5-a8dfccec128c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:14:36.941091 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (654af877-a828-430e-84a0-8ee6cc4e8867) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:36.994399 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (654af877-a828-430e-84a0-8ee6cc4e8867) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:14:36Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_c91269ce-6685-486d-80a1-f85db4d9cc4e',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2ff1c774-21f8-4132-bc9e-efb38d1bc9f3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='7e0169496f8740ca83bc6f92594ca95d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:14:37Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:14:37Z,use_quota=True,user_id='0042863b26ca408a9ab608edba9951ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:37.005202 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7788edfe-5bd3-45b9-bf4f-e815d6cd2ecc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:37.010844 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7788edfe-5bd3-45b9-bf4f-e815d6cd2ecc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:37.024549 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8652cfff-d8f2-467b-a971-dcb63a43469f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:37.043244 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8652cfff-d8f2-467b-a971-dcb63a43469f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2ff1c774-21f8-4132-bc9e-efb38d1bc9f3', 'volume_name': 'volume-2ff1c774-21f8-4132-bc9e-efb38d1bc9f3', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:37.059752 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b73a05a9-8177-4c81-9f5a-ad9aff4eac78) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:37.071038 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b73a05a9-8177-4c81-9f5a-ad9aff4eac78) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:37.087581 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d6740edd-baf0-4d7f-ae87-50043dcd51aa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:37.088828 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Volume 2ff1c774-21f8-4132-bc9e-efb38d1bc9f3: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2ff1c774-21f8-4132-bc9e-efb38d1bc9f3', 'volume_size': 1} Sep 17 21:14:37.089236 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] creating volume 'volume-2ff1c774-21f8-4132-bc9e-efb38d1bc9f3' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:14:37.096208 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:37.238512 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:14:37.239391 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:14:37.243885 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d6740edd-baf0-4d7f-ae87-50043dcd51aa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2ff1c774-21f8-4132-bc9e-efb38d1bc9f3', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:37.253538 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7b73b7c7-0a9c-4f63-97ce-f8992f6b4eb3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:37.276679 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Volume volume-2ff1c774-21f8-4132-bc9e-efb38d1bc9f3 (2ff1c774-21f8-4132-bc9e-efb38d1bc9f3): created successfully Sep 17 21:14:37.287312 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7b73b7c7-0a9c-4f63-97ce-f8992f6b4eb3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:37.304421 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Flow 'volume_create_manager' (570b2fc6-1f86-4c26-9ae5-a8dfccec128c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:14:37.317974 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Created volume successfully. Sep 17 21:14:37.931114 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:14:38.904592 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.972s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:14:38.906627 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2ff1c774-21f8-4132-bc9e-efb38d1bc9f3', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '2ff1c774-21f8-4132-bc9e-efb38d1bc9f3', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:14:38.932040 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Initialize volume connection completed successfully. Sep 17 21:14:40.973035 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Terminate volume connection completed successfully. Sep 17 21:14:41.003747 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7928dc82-5f2c-4010-954b-da005af7c663 tempest-VolumesBackupsAdminTest-841724568 None] Remove volume export completed successfully. Sep 17 21:14:42.295356 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-74ef6d8f-3ce9-43e9-ad59-5a54292a89b3 tempest-VolumesBackupsAdminTest-841724568 None] Acquiring lock "cinder-2ff1c774-21f8-4132-bc9e-efb38d1bc9f3-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:14:42.312860 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-74ef6d8f-3ce9-43e9-ad59-5a54292a89b3 tempest-VolumesBackupsAdminTest-841724568 None] Lock "cinder-2ff1c774-21f8-4132-bc9e-efb38d1bc9f3-delete_volume" acquired by "delete_volume" :: waited 0.017s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:14:42.359622 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=62,cinder:INSERT=6,cinder:UPDATE=30,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:42.380822 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-74ef6d8f-3ce9-43e9-ad59-5a54292a89b3 tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:42.627411 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-74ef6d8f-3ce9-43e9-ad59-5a54292a89b3 tempest-VolumesBackupsAdminTest-841724568 None] Volume volume-2ff1c774-21f8-4132-bc9e-efb38d1bc9f3 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:14:42.667806 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-74ef6d8f-3ce9-43e9-ad59-5a54292a89b3 tempest-VolumesBackupsAdminTest-841724568 None] deleting rbd volume volume-2ff1c774-21f8-4132-bc9e-efb38d1bc9f3 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:14:43.046692 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=63,cinder:UPDATE=28,cinder:INSERT=6,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:14:43.084599 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-74ef6d8f-3ce9-43e9-ad59-5a54292a89b3 tempest-VolumesBackupsAdminTest-841724568 None] Created reservations ['ed0c74a9-b118-4a80-942a-f98cdef80e7e', 'd58472da-7384-4ac1-a106-65916e6bffb4', 'f91e3aae-e614-4d78-8171-c4233cfaa1bd', '9411089a-25be-4e85-a61a-801041849910'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:14:43.164000 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-74ef6d8f-3ce9-43e9-ad59-5a54292a89b3 tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:43.241553 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-74ef6d8f-3ce9-43e9-ad59-5a54292a89b3 tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:43.308161 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-74ef6d8f-3ce9-43e9-ad59-5a54292a89b3 tempest-VolumesBackupsAdminTest-841724568 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:14:43.322035 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-74ef6d8f-3ce9-43e9-ad59-5a54292a89b3 tempest-VolumesBackupsAdminTest-841724568 None] Deleted volume successfully. Sep 17 21:14:43.345343 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-74ef6d8f-3ce9-43e9-ad59-5a54292a89b3 tempest-VolumesBackupsAdminTest-841724568 None] Lock "cinder-2ff1c774-21f8-4132-bc9e-efb38d1bc9f3-delete_volume" released by "delete_volume" :: held 1.032s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:14:46.510123 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:14:46.564642 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Flow 'volume_create_manager' (7aa31d19-fb48-43f9-8c6d-cedd9002d683) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:14:46.572757 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (778a478d-248e-4312-b3e1-a54759a5e1c8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:46.596790 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (778a478d-248e-4312-b3e1-a54759a5e1c8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:14:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Volume-1644273723',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=294b0d1e-5c23-404d-906e-4877cbac9b96,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='7e0169496f8740ca83bc6f92594ca95d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:14:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:14:46Z,use_quota=True,user_id='0042863b26ca408a9ab608edba9951ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:46.609232 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0cac4d5f-d557-425e-976e-8da4d9f06024) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:46.626001 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0cac4d5f-d557-425e-976e-8da4d9f06024) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:46.632405 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a2446d99-a9f4-4f29-a386-78758873a5c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:46.639588 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a2446d99-a9f4-4f29-a386-78758873a5c7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '294b0d1e-5c23-404d-906e-4877cbac9b96', 'volume_name': 'volume-294b0d1e-5c23-404d-906e-4877cbac9b96', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:46.645024 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7842da02-93f9-4490-b4eb-5c5df205b73b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:46.649376 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7842da02-93f9-4490-b4eb-5c5df205b73b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:46.655146 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b64a9d0f-56aa-44cd-9fb3-15e0d7842bc3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:46.657177 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Volume 294b0d1e-5c23-404d-906e-4877cbac9b96: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-294b0d1e-5c23-404d-906e-4877cbac9b96', 'volume_size': 1} Sep 17 21:14:46.657820 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] creating volume 'volume-294b0d1e-5c23-404d-906e-4877cbac9b96' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:14:46.658890 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:46.770857 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:14:46.771837 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:14:46.776656 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b64a9d0f-56aa-44cd-9fb3-15e0d7842bc3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-294b0d1e-5c23-404d-906e-4877cbac9b96', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:46.782905 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (57ccb086-d409-4ca6-9c86-c294f7fbd768) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:14:46.798431 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Volume volume-294b0d1e-5c23-404d-906e-4877cbac9b96 (294b0d1e-5c23-404d-906e-4877cbac9b96): created successfully Sep 17 21:14:46.812316 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (57ccb086-d409-4ca6-9c86-c294f7fbd768) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:14:46.824855 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Flow 'volume_create_manager' (7aa31d19-fb48-43f9-8c6d-cedd9002d683) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:14:46.859019 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-af5f4f32-3ffa-4c5b-80ed-11a085253dc0 tempest-VolumesBackupsAdminTest-841724568 None] Created volume successfully. Sep 17 21:14:47.662360 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ba486445-9ccc-45bc-ba60-6967a36372b9 tempest-VolumesBackupsAdminTest-841724568 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-17T21:14:47Z,data_timestamp=2022-09-17T21:14:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-1436798716',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=0b03f26e-af35-463f-9a25-3cf84edba690,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='7e0169496f8740ca83bc6f92594ca95d',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='0042863b26ca408a9ab608edba9951ac',volume_id=294b0d1e-5c23-404d-906e-4877cbac9b96) Sep 17 21:14:47.663133 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.backup.rpcapi [None req-ba486445-9ccc-45bc-ba60-6967a36372b9 tempest-VolumesBackupsAdminTest-841724568 None] continue_backup in rpcapi backup_id 0b03f26e-af35-463f-9a25-3cf84edba690 {{(pid=133409) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 17 21:14:47.842439 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-ba486445-9ccc-45bc-ba60-6967a36372b9 tempest-VolumesBackupsAdminTest-841724568 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:14:48.987595 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-ba486445-9ccc-45bc-ba60-6967a36372b9 tempest-VolumesBackupsAdminTest-841724568 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.144s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:14:48.989692 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ba486445-9ccc-45bc-ba60-6967a36372b9 tempest-VolumesBackupsAdminTest-841724568 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-294b0d1e-5c23-404d-906e-4877cbac9b96', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '294b0d1e-5c23-404d-906e-4877cbac9b96', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:14:49.037753 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ba486445-9ccc-45bc-ba60-6967a36372b9 tempest-VolumesBackupsAdminTest-841724568 None] Initialize volume connection completed successfully. Sep 17 21:14:51.109425 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ba486445-9ccc-45bc-ba60-6967a36372b9 tempest-VolumesBackupsAdminTest-841724568 None] Terminate volume connection completed successfully. Sep 17 21:14:51.146850 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ba486445-9ccc-45bc-ba60-6967a36372b9 tempest-VolumesBackupsAdminTest-841724568 None] Remove volume export completed successfully. Sep 17 21:14:54.540882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d13678eb-c91b-4b07-ba70-a7214acf041c tempest-VolumesBackupsAdminTest-841724568 None] Acquiring lock "cinder-294b0d1e-5c23-404d-906e-4877cbac9b96-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:14:54.576075 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d13678eb-c91b-4b07-ba70-a7214acf041c tempest-VolumesBackupsAdminTest-841724568 None] Lock "cinder-294b0d1e-5c23-404d-906e-4877cbac9b96-delete_volume" acquired by "delete_volume" :: waited 0.039s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:14:54.660952 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d13678eb-c91b-4b07-ba70-a7214acf041c tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:55.637453 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d13678eb-c91b-4b07-ba70-a7214acf041c tempest-VolumesBackupsAdminTest-841724568 None] Volume volume-294b0d1e-5c23-404d-906e-4877cbac9b96 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:14:55.662041 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d13678eb-c91b-4b07-ba70-a7214acf041c tempest-VolumesBackupsAdminTest-841724568 None] deleting rbd volume volume-294b0d1e-5c23-404d-906e-4877cbac9b96 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:14:56.130711 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-d13678eb-c91b-4b07-ba70-a7214acf041c tempest-VolumesBackupsAdminTest-841724568 None] Created reservations ['52c89095-2fd4-4e2b-974b-fa4b68d7ec81', 'b666e66c-d7ce-480a-b798-ff6fbfbe2cc9', '2ee31671-155c-4932-927c-f420164271a9', '62827e5a-006f-4c81-abdc-71ff3996999c'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:14:56.206200 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d13678eb-c91b-4b07-ba70-a7214acf041c tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:56.284036 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d13678eb-c91b-4b07-ba70-a7214acf041c tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:56.360788 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-d13678eb-c91b-4b07-ba70-a7214acf041c tempest-VolumesBackupsAdminTest-841724568 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:14:56.365452 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d13678eb-c91b-4b07-ba70-a7214acf041c tempest-VolumesBackupsAdminTest-841724568 None] Deleted volume successfully. Sep 17 21:14:56.391507 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d13678eb-c91b-4b07-ba70-a7214acf041c tempest-VolumesBackupsAdminTest-841724568 None] Lock "cinder-294b0d1e-5c23-404d-906e-4877cbac9b96-delete_volume" released by "delete_volume" :: held 1.815s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:14:56.924577 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-560fd6e1-13ba-4d80-8e1a-0c184960488b tempest-VolumesBackupsAdminTest-841724568 None] Acquiring lock "cinder-4adec8bd-67e4-4d63-8177-46e0243e55c6-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:14:56.941357 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-560fd6e1-13ba-4d80-8e1a-0c184960488b tempest-VolumesBackupsAdminTest-841724568 None] Lock "cinder-4adec8bd-67e4-4d63-8177-46e0243e55c6-delete_volume" acquired by "delete_volume" :: waited 0.017s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:14:57.000278 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-560fd6e1-13ba-4d80-8e1a-0c184960488b tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:57.768029 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-560fd6e1-13ba-4d80-8e1a-0c184960488b tempest-VolumesBackupsAdminTest-841724568 None] Volume volume-4adec8bd-67e4-4d63-8177-46e0243e55c6 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:14:57.796544 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-560fd6e1-13ba-4d80-8e1a-0c184960488b tempest-VolumesBackupsAdminTest-841724568 None] deleting rbd volume volume-4adec8bd-67e4-4d63-8177-46e0243e55c6 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:14:58.309350 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-560fd6e1-13ba-4d80-8e1a-0c184960488b tempest-VolumesBackupsAdminTest-841724568 None] Created reservations ['3ddcb35e-52f9-46ca-b561-8a5a890fad93', '4e675eb4-09d6-4527-8117-97c328646ac0', 'edf1fb9d-a851-414e-ab63-3e92068f6757', '5259c205-7dfe-4c94-960b-243c00aa47dd'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:14:58.409289 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-560fd6e1-13ba-4d80-8e1a-0c184960488b tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:58.483197 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-560fd6e1-13ba-4d80-8e1a-0c184960488b tempest-VolumesBackupsAdminTest-841724568 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:14:58.549358 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-560fd6e1-13ba-4d80-8e1a-0c184960488b tempest-VolumesBackupsAdminTest-841724568 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:14:58.554351 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-560fd6e1-13ba-4d80-8e1a-0c184960488b tempest-VolumesBackupsAdminTest-841724568 None] Deleted volume successfully. Sep 17 21:14:58.579321 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-560fd6e1-13ba-4d80-8e1a-0c184960488b tempest-VolumesBackupsAdminTest-841724568 None] Lock "cinder-4adec8bd-67e4-4d63-8177-46e0243e55c6-delete_volume" released by "delete_volume" :: held 1.637s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:15:01.031008 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:15:01.034982 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:01.159280 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:01.254735 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:15:06.247806 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:15:06.304118 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Flow 'volume_create_manager' (b6947252-6740-4ec6-b043-7d0592f2c690) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:06.330238 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bf1d3afc-d904-4652-ad18-608d33895f7f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:06.365649 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bf1d3afc-d904-4652-ad18-608d33895f7f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:15:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesImageMetadata-Volume-722904581',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bfeaab1c-76ff-4769-9354-d35207547695,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='dbeef98f1fb44c26a13741e637a819d7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:15:06Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:15:06Z,use_quota=True,user_id='0e63bded930641d8a15958628cf9a7aa',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:06.379580 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (acf87890-485e-4a72-a149-bf61f11de4dc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:06.389032 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (acf87890-485e-4a72-a149-bf61f11de4dc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:06.394356 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (438bea68-0f31-4ca8-89c4-175ecf500628) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:06.549057 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (438bea68-0f31-4ca8-89c4-175ecf500628) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'bfeaab1c-76ff-4769-9354-d35207547695', 'volume_name': 'volume-bfeaab1c-76ff-4769-9354-d35207547695', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:06.556196 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (69712efe-295e-46f5-bcfb-79564299e077) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:06.560443 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (69712efe-295e-46f5-bcfb-79564299e077) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:06.566391 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3f18fd0a-de1b-4ed3-a8cb-40745c485136) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:06.568419 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Volume bfeaab1c-76ff-4769-9354-d35207547695: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-bfeaab1c-76ff-4769-9354-d35207547695', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 21:15:06.569281 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Cloning bfeaab1c-76ff-4769-9354-d35207547695 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 21:15:06.570635 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:15:06.581529 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.011s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:15:06.592786 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=55,cinder:INSERT=12,cinder:UPDATE=30,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:06.598687 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 15, 6, 587348)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:15:06.599472 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:15:06.600080 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 21:15:06.615936 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.033s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:15:06.617496 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 21:15:06.632543 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 15, 6, 628203)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:15:06.633294 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:15:06.634509 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 21:15:06.656537 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:06.763143 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] creating snapshot='volume-bfeaab1c-76ff-4769-9354-d35207547695.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:15:07.730376 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-bfeaab1c-76ff-4769-9354-d35207547695.clone_snap' to 'volume-bfeaab1c-76ff-4769-9354-d35207547695' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:15:07.968199 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:15:07.969718 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:15:07.970513 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:15:07.991715 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:15:07.992476 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Marking volume bfeaab1c-76ff-4769-9354-d35207547695 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:15:08.004008 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to bfeaab1c-76ff-4769-9354-d35207547695. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 21:15:08.004461 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Creating volume glance metadata for volume bfeaab1c-76ff-4769-9354-d35207547695 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 21:15:08.126409 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3f18fd0a-de1b-4ed3-a8cb-40745c485136) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bfeaab1c-76ff-4769-9354-d35207547695', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:08.134774 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2068a5c8-450a-412a-a61f-f618fc4ae988) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:08.151611 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Volume volume-bfeaab1c-76ff-4769-9354-d35207547695 (bfeaab1c-76ff-4769-9354-d35207547695): created successfully Sep 17 21:15:08.158083 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2068a5c8-450a-412a-a61f-f618fc4ae988) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:08.170678 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Flow 'volume_create_manager' (b6947252-6740-4ec6-b043-7d0592f2c690) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:08.200891 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b96a2c8b-d4ca-46dd-85a2-b428c65a881a tempest-VolumesImageMetadata-943335724 None] Created volume successfully. Sep 17 21:15:08.897092 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] Acquiring lock "cinder-bfeaab1c-76ff-4769-9354-d35207547695-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:15:08.927444 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] Lock "cinder-bfeaab1c-76ff-4769-9354-d35207547695-delete_volume" acquired by "delete_volume" :: waited 0.028s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:15:08.995453 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:09.135900 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:15:09.154698 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] deleting rbd volume volume-bfeaab1c-76ff-4769-9354-d35207547695 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:15:09.537361 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:15:09.577391 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] deleting parent snapshot volume-bfeaab1c-76ff-4769-9354-d35207547695.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:15:10.832188 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=66,cinder:UPDATE=27,cinder:DELETE=1,cinder:INSERT=6 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:10.862280 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] Created reservations ['d2b9de4e-c271-4d1b-90fa-5413d8e19c41', 'a1c86c9c-0fee-4ed2-8a0f-6165c46c887a', '3e596cf9-c22b-4f0e-aede-7b7f44f39b57', '26c63272-7667-4bbb-88bc-45b87962a86d'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:15:10.972761 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:11.070471 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:11.131229 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:15:11.136496 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] Deleted volume successfully. Sep 17 21:15:11.157739 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-746539bb-b17a-4c38-80dc-fbe2545c5c2c tempest-VolumesImageMetadata-943335724 None] Lock "cinder-bfeaab1c-76ff-4769-9354-d35207547695-delete_volume" released by "delete_volume" :: held 2.232s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:15:14.511320 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:15:14.552607 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Flow 'volume_create_manager' (b9af89d7-d8c6-4e28-b795-b22c45249e11) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:14.569438 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9161292f-b75a-411c-8164-4bd6e32f56e0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:14.612035 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9161292f-b75a-411c-8164-4bd6e32f56e0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:15:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-190604461',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b2bc9cfa-f40b-411e-8a44-20b3f6134615,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='720881c6f7304aadbf069698c97dc5c5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:15:14Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:15:14Z,use_quota=True,user_id='8ddb1d0a7bca457493d7ac9a58a01e64',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:14.629495 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (bff3cd36-8fef-44f8-bed9-50aaf0d70a75) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:14.635360 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (bff3cd36-8fef-44f8-bed9-50aaf0d70a75) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:14.649228 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (86ae922a-209b-4eba-9753-d8ba787fb60a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:14.674891 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (86ae922a-209b-4eba-9753-d8ba787fb60a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b2bc9cfa-f40b-411e-8a44-20b3f6134615', 'volume_name': 'volume-b2bc9cfa-f40b-411e-8a44-20b3f6134615', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:14.681794 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5609ca46-91f2-4102-91be-d9dd4edb0205) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:14.686377 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5609ca46-91f2-4102-91be-d9dd4edb0205) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:14.702174 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b92dfd40-1b56-4ed8-93fc-57f3143e4fc5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:14.703583 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Volume b2bc9cfa-f40b-411e-8a44-20b3f6134615: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b2bc9cfa-f40b-411e-8a44-20b3f6134615', 'volume_size': 1} Sep 17 21:15:14.706316 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] creating volume 'volume-b2bc9cfa-f40b-411e-8a44-20b3f6134615' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:15:14.708364 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:14.886260 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:15:14.886946 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:15:14.892339 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b92dfd40-1b56-4ed8-93fc-57f3143e4fc5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b2bc9cfa-f40b-411e-8a44-20b3f6134615', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:14.902883 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c9be0e90-b5d5-4cba-bc1f-89639f2f1f69) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:14.932492 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Volume volume-b2bc9cfa-f40b-411e-8a44-20b3f6134615 (b2bc9cfa-f40b-411e-8a44-20b3f6134615): created successfully Sep 17 21:15:14.941965 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c9be0e90-b5d5-4cba-bc1f-89639f2f1f69) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:14.959258 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Flow 'volume_create_manager' (b9af89d7-d8c6-4e28-b795-b22c45249e11) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:14.980666 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c9fff978-a0e3-4597-b0ac-f2c56634195c tempest-AttachVolumeShelveTestJSON-1971975439 None] Created volume successfully. Sep 17 21:15:17.072441 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:15:17.108424 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Flow 'volume_create_manager' (ed5e7202-b8fd-463a-af37-5539f68f2427) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:17.117410 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0c13d5d7-b73a-4a42-a90a-b26e4bd06fc2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:17.137736 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0c13d5d7-b73a-4a42-a90a-b26e4bd06fc2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:15:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AbsoluteLimitsTests-Volume-193564571',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8d5604ee-c511-4f84-b8f1-e0c0c84516d9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a7cfffdc05394baaa8a064b5570772c6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:15:17Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:15:17Z,use_quota=True,user_id='17bdd318ee7a4f6ba3092804c2ff40bf',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:17.144629 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b690048a-2992-419d-aaab-0a61b4867799) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:17.163763 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b690048a-2992-419d-aaab-0a61b4867799) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:17.182352 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (82c994a4-63e9-4797-9c17-6e6eb133e14e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:17.199251 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (82c994a4-63e9-4797-9c17-6e6eb133e14e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8d5604ee-c511-4f84-b8f1-e0c0c84516d9', 'volume_name': 'volume-8d5604ee-c511-4f84-b8f1-e0c0c84516d9', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:17.208079 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (aab6b231-05ea-4b3e-a424-f75261304f0c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:17.223567 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (aab6b231-05ea-4b3e-a424-f75261304f0c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:17.228687 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d4a201be-eecb-4874-9012-d2443c748ee1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:17.235968 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Volume 8d5604ee-c511-4f84-b8f1-e0c0c84516d9: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8d5604ee-c511-4f84-b8f1-e0c0c84516d9', 'volume_size': 1} Sep 17 21:15:17.236338 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] creating volume 'volume-8d5604ee-c511-4f84-b8f1-e0c0c84516d9' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:15:17.241791 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:17.458850 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:15:17.459958 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:15:17.464225 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d4a201be-eecb-4874-9012-d2443c748ee1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8d5604ee-c511-4f84-b8f1-e0c0c84516d9', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:17.476088 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c012fe72-b547-49eb-80f0-72e703994220) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:17.504053 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Volume volume-8d5604ee-c511-4f84-b8f1-e0c0c84516d9 (8d5604ee-c511-4f84-b8f1-e0c0c84516d9): created successfully Sep 17 21:15:17.508307 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c012fe72-b547-49eb-80f0-72e703994220) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:17.514653 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Flow 'volume_create_manager' (ed5e7202-b8fd-463a-af37-5539f68f2427) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:17.528735 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-94f866b5-5a3a-4c88-a1bb-e20a68c55cd1 tempest-AbsoluteLimitsTests-1633990675 None] Created volume successfully. Sep 17 21:15:18.323516 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] Acquiring lock "cinder-8d5604ee-c511-4f84-b8f1-e0c0c84516d9-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:15:18.349430 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] Lock "cinder-8d5604ee-c511-4f84-b8f1-e0c0c84516d9-delete_volume" acquired by "delete_volume" :: waited 0.027s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:15:18.439870 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:18.511129 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:15:18.511559 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] Volume volume-8d5604ee-c511-4f84-b8f1-e0c0c84516d9 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:15:18.520316 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] deleting rbd volume volume-8d5604ee-c511-4f84-b8f1-e0c0c84516d9 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:15:18.938130 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] Created reservations ['7a420a11-a03e-4eb8-a3ef-fc9eb6865c28', '8629410d-f48c-4dde-acfc-3fa0e3766d1a', '2c0b9fcf-080a-41ee-93a9-9d35ea8c5ffd', '2aeedf66-efb0-4658-87cc-2e48b22332e2'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:15:19.065153 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:19.138897 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:19.217400 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:15:19.225292 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] Deleted volume successfully. Sep 17 21:15:19.249799 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6bca8eac-6ccf-4baa-93d6-c539c5eecac5 tempest-AbsoluteLimitsTests-1633990675 None] Lock "cinder-8d5604ee-c511-4f84-b8f1-e0c0c84516d9-delete_volume" released by "delete_volume" :: held 0.900s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:15:23.320475 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-716f7cc4-5bf8-4755-b511-b9f7b5ec09ab tempest-GroupsV314Test-1755171075 None] Group tempest-GroupsV314Test-Group-852885048: creating Sep 17 21:15:23.394499 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-716f7cc4-5bf8-4755-b511-b9f7b5ec09ab tempest-GroupsV314Test-1755171075 None] Group tempest-GroupsV314Test-Group-852885048: created successfully Sep 17 21:15:23.411138 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-716f7cc4-5bf8-4755-b511-b9f7b5ec09ab tempest-GroupsV314Test-1755171075 None] Create group completed successfully. Sep 17 21:15:25.492625 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:15:25.549901 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Flow 'volume_create_manager' (13904c67-0a30-4545-98e5-9123f38313c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:25.568925 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2956bc30-9d61-4096-a6a2-90cbdc93e844) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:25.605996 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2956bc30-9d61-4096-a6a2-90cbdc93e844) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:15:25Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsV314Test-Volume-2134219148',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=696f35c0-e14d-4f7c-9e09-b3b41d783714,host='ncdv-master-1@ceph#ceph',id=1dc75056-8649-4ab9-8df2-f58bc827b5bf,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='bf380d9339814239b9e1078b99c4905a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:15:25Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:15:25Z,use_quota=True,user_id='9b96d4611d66470082a5574fde431b3c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(a8b7f892-32ef-4af3-8da9-c759814c9ef6),volume_type_id=a8b7f892-32ef-4af3-8da9-c759814c9ef6)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:25.622563 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f7cf9561-7e8f-41d7-8eb6-8c146ab88f41) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:25.626924 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f7cf9561-7e8f-41d7-8eb6-8c146ab88f41) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:25.652925 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c0218acb-14b0-41f8-be98-ea0978ca2873) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:25.678307 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c0218acb-14b0-41f8-be98-ea0978ca2873) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1dc75056-8649-4ab9-8df2-f58bc827b5bf', 'volume_name': 'volume-1dc75056-8649-4ab9-8df2-f58bc827b5bf', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:25.692600 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4e6d49d7-b01c-4b77-86c2-cba16ed809de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:25.700609 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4e6d49d7-b01c-4b77-86c2-cba16ed809de) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:25.707330 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2800af01-d3c0-4aba-af69-1fe312523987) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:25.708888 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Volume 1dc75056-8649-4ab9-8df2-f58bc827b5bf: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1dc75056-8649-4ab9-8df2-f58bc827b5bf', 'volume_size': 1} Sep 17 21:15:25.709510 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] creating volume 'volume-1dc75056-8649-4ab9-8df2-f58bc827b5bf' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:15:25.710514 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:25.854634 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:15:25.855267 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:15:25.859429 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2800af01-d3c0-4aba-af69-1fe312523987) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1dc75056-8649-4ab9-8df2-f58bc827b5bf', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:25.867733 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e09e380d-4cc9-4d40-b33d-0dd73c20a828) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:25.895054 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Volume volume-1dc75056-8649-4ab9-8df2-f58bc827b5bf (1dc75056-8649-4ab9-8df2-f58bc827b5bf): created successfully Sep 17 21:15:25.900192 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e09e380d-4cc9-4d40-b33d-0dd73c20a828) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:25.908562 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Flow 'volume_create_manager' (13904c67-0a30-4545-98e5-9123f38313c5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:25.925448 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-46229c6c-447c-465f-b0dc-77c048d9e6f3 tempest-GroupsV314Test-1755171075 None] Created volume successfully. Sep 17 21:15:27.342692 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cefb7716-774b-4985-b146-90f536eab089 tempest-GroupsV314Test-1755171075 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:27.455325 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cefb7716-774b-4985-b146-90f536eab089 tempest-GroupsV314Test-1755171075 None] creating snapshot='volume-c53ed7ea-2dfa-4a94-aad8-f9c3582b698c.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:15:27.999336 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cefb7716-774b-4985-b146-90f536eab089 tempest-GroupsV314Test-1755171075 None] cloning 'volume-1dc75056-8649-4ab9-8df2-f58bc827b5bf@volume-c53ed7ea-2dfa-4a94-aad8-f9c3582b698c.clone_snap' to 'volume-c53ed7ea-2dfa-4a94-aad8-f9c3582b698c' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:15:28.182687 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cefb7716-774b-4985-b146-90f536eab089 tempest-GroupsV314Test-1755171075 None] Volume volume-1dc75056-8649-4ab9-8df2-f58bc827b5bf is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:15:28.193958 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cefb7716-774b-4985-b146-90f536eab089 tempest-GroupsV314Test-1755171075 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:15:28.194736 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cefb7716-774b-4985-b146-90f536eab089 tempest-GroupsV314Test-1755171075 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:15:28.220576 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cefb7716-774b-4985-b146-90f536eab089 tempest-GroupsV314Test-1755171075 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:15:28.260290 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:15:28.281600 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Flow 'volume_create_manager' (5fd25112-1091-4799-b48c-cf17c38c7d98) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:28.306889 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f0bd11a2-f910-414c-8a97-506ab2fc2e6b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:28.330247 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f0bd11a2-f910-414c-8a97-506ab2fc2e6b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:15:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV355Test-Volume-1437848249',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a256e83c-a088-46bd-8965-1acaf022a044,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d485f1219d7f471cba8ebface38c8cec',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:15:28Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:15:28Z,use_quota=True,user_id='5f861462391648188d43f16bd9348979',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:28.336370 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7a1ffe83-c484-40b4-a514-c585f4f3f668) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:28.341528 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7a1ffe83-c484-40b4-a514-c585f4f3f668) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:28.346854 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ad2b134b-e6ab-4625-ba25-7e9262aeecbb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:28.351873 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ad2b134b-e6ab-4625-ba25-7e9262aeecbb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a256e83c-a088-46bd-8965-1acaf022a044', 'volume_name': 'volume-a256e83c-a088-46bd-8965-1acaf022a044', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:28.357018 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (da58ba92-de3b-4d0b-889b-600365caee96) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:28.361354 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (da58ba92-de3b-4d0b-889b-600365caee96) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:28.366386 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (14ce4444-726a-4445-9236-0844ec853cf3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:28.367660 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Volume a256e83c-a088-46bd-8965-1acaf022a044: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a256e83c-a088-46bd-8965-1acaf022a044', 'volume_size': 1} Sep 17 21:15:28.368985 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] creating volume 'volume-a256e83c-a088-46bd-8965-1acaf022a044' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:15:28.369832 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:28.457691 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-cefb7716-774b-4985-b146-90f536eab089 tempest-GroupsV314Test-1755171075 None] Create group from source-group-696f35c0-e14d-4f7c-9e09-b3b41d783714 completed successfully. Sep 17 21:15:28.466974 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:15:28.467592 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:15:28.473036 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (14ce4444-726a-4445-9236-0844ec853cf3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a256e83c-a088-46bd-8965-1acaf022a044', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:28.479102 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d6d36308-9765-4dd3-a5ee-1c107d407023) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:28.495380 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Volume volume-a256e83c-a088-46bd-8965-1acaf022a044 (a256e83c-a088-46bd-8965-1acaf022a044): created successfully Sep 17 21:15:28.500976 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d6d36308-9765-4dd3-a5ee-1c107d407023) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:28.519395 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Flow 'volume_create_manager' (5fd25112-1091-4799-b48c-cf17c38c7d98) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:28.536368 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d9d3b59d-e8d2-4f30-9487-a7e90c9c5cf4 tempest-VolumesTransfersV355Test-1097059843 None] Created volume successfully. Sep 17 21:15:29.525474 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=55,cinder:UPDATE=27,cinder:INSERT=13,cinder:DELETE=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:29.551924 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:29.667550 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:15:29.701657 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] deleting rbd volume volume-c53ed7ea-2dfa-4a94-aad8-f9c3582b698c {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:15:30.124984 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:15:30.153535 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] Volume volume-1dc75056-8649-4ab9-8df2-f58bc827b5bf is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:15:30.154226 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] deleting parent snapshot volume-c53ed7ea-2dfa-4a94-aad8-f9c3582b698c.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:15:31.156601 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5a1b1c58-6d94-4b79-af89-124f65b3aa6b tempest-VolumesTransfersV355Test-771720448 None] Transfer volume completed successfully. Sep 17 21:15:31.284488 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] Created reservations ['87caabb0-ae02-44f1-8927-7017f8a64b87'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:15:31.348839 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=27,cinder:SELECT=70,cinder:DELETE=2,cinder:INSERT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:31.353320 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] Deprecated: Default quota for resource: volumes_tempest-GroupsV314Test-volume-type-1158579321 is set by the default quota flag: quota_volumes_tempest-GroupsV314Test-volume-type-1158579321, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:15:31.354491 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsV314Test-volume-type-1158579321 is set by the default quota flag: quota_gigabytes_tempest-GroupsV314Test-volume-type-1158579321, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:15:31.442688 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] Created reservations ['eca7d73a-fc98-4ce0-9349-9c95f7242412', 'a8a8ac9c-fbd9-4bb7-9dd7-1ab8d54056b1', '7474a79c-39af-4116-a00b-8ceb16be48dc', '8a7bbb26-d7c1-4605-b5af-abedecf2375d'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:15:31.633959 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:31.758042 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:31.871006 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:15:31.892764 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8257f713-74be-4dec-a1f3-6fe9a11a7368 tempest-GroupsV314Test-1755171075 None] Delete group completed successfully. Sep 17 21:15:32.003237 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] Acquiring lock "cinder-a256e83c-a088-46bd-8965-1acaf022a044-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:15:32.016694 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] Lock "cinder-a256e83c-a088-46bd-8965-1acaf022a044-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:15:32.138705 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:32.274078 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:15:32.275267 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] Volume volume-a256e83c-a088-46bd-8965-1acaf022a044 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:15:32.283946 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] deleting rbd volume volume-a256e83c-a088-46bd-8965-1acaf022a044 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:15:32.451358 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-37223bb5-0d9f-4955-acdc-0a63316556a1 tempest-GroupsV314Test-1755171075 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:32.554484 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-37223bb5-0d9f-4955-acdc-0a63316556a1 tempest-GroupsV314Test-1755171075 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:15:32.554859 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-37223bb5-0d9f-4955-acdc-0a63316556a1 tempest-GroupsV314Test-1755171075 None] Volume volume-1dc75056-8649-4ab9-8df2-f58bc827b5bf is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:15:32.569156 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-37223bb5-0d9f-4955-acdc-0a63316556a1 tempest-GroupsV314Test-1755171075 None] deleting rbd volume volume-1dc75056-8649-4ab9-8df2-f58bc827b5bf {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:15:32.760664 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] Created reservations ['108c7718-1be4-40af-9f5d-3fb2cf2e1858', 'e8f8f196-453d-4630-9e5b-73633680ad7d', '815c745e-e6f4-46f8-8484-77a1d81bee2f', '3f9b707a-12ac-4927-9eee-50fd872c408c'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:15:32.879199 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:33.006475 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:33.106374 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:15:33.113910 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] Deleted volume successfully. Sep 17 21:15:33.170267 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b3118835-bf26-4b20-ae8a-c6b9ef8f5d52 tempest-VolumesTransfersV355Test-287664992 None] Lock "cinder-a256e83c-a088-46bd-8965-1acaf022a044-delete_volume" released by "delete_volume" :: held 1.154s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:15:33.196482 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-37223bb5-0d9f-4955-acdc-0a63316556a1 tempest-GroupsV314Test-1755171075 None] Created reservations ['c6c79ca5-8973-4847-8cd4-c8299c3084db'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:15:33.278114 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-37223bb5-0d9f-4955-acdc-0a63316556a1 tempest-GroupsV314Test-1755171075 None] Created reservations ['9a6f7636-99ec-4421-ab13-a27f2b873c0c', 'a80375e6-05e5-4580-9913-6bda894f2fce', 'fe83d6ee-9921-4797-bbea-d43dfc38a841', 'e1629933-a0ca-46d7-8840-6f4f625831d2'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:15:33.495731 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-37223bb5-0d9f-4955-acdc-0a63316556a1 tempest-GroupsV314Test-1755171075 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:33.618428 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-37223bb5-0d9f-4955-acdc-0a63316556a1 tempest-GroupsV314Test-1755171075 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:33.718742 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-37223bb5-0d9f-4955-acdc-0a63316556a1 tempest-GroupsV314Test-1755171075 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:15:33.724964 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-37223bb5-0d9f-4955-acdc-0a63316556a1 tempest-GroupsV314Test-1755171075 None] Delete group completed successfully. Sep 17 21:15:34.148613 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:15:34.189512 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Flow 'volume_create_manager' (de20457a-d1e7-4e56-8b03-f8f38a2eab3c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:34.212107 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (962a79aa-aacd-4b0e-944b-f60ed1e61f77) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:34.247399 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (962a79aa-aacd-4b0e-944b-f60ed1e61f77) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:15:34Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV355Test-Volume-2055043675',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d485f1219d7f471cba8ebface38c8cec',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:15:34Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:15:34Z,use_quota=True,user_id='5f861462391648188d43f16bd9348979',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:34.263488 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (80db10e3-65e3-4ff6-bb9f-7a6f0fefd8c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:34.267857 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (80db10e3-65e3-4ff6-bb9f-7a6f0fefd8c7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:34.287478 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (024f27ab-b547-4e53-8a2b-8b245a6fac75) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:34.302809 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (024f27ab-b547-4e53-8a2b-8b245a6fac75) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7', 'volume_name': 'volume-7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:34.316417 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1e56375f-b6dc-432d-9fb1-0a21cf2c594b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:34.325913 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1e56375f-b6dc-432d-9fb1-0a21cf2c594b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:34.336218 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c79a0df6-9d12-4b41-871b-c9e8680638e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:34.339853 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Volume 7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7', 'volume_size': 1} Sep 17 21:15:34.341591 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] creating volume 'volume-7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:15:34.342698 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:34.469400 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:15:34.470052 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:15:34.474024 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c79a0df6-9d12-4b41-871b-c9e8680638e5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:34.485830 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (51f6c242-ad26-4a6f-8926-36240a51ce6b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:34.508505 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Volume volume-7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7 (7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7): created successfully Sep 17 21:15:34.512283 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (51f6c242-ad26-4a6f-8926-36240a51ce6b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:34.517873 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Flow 'volume_create_manager' (de20457a-d1e7-4e56-8b03-f8f38a2eab3c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:34.547117 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c21d0faa-a756-4b21-bd22-6488b5764afb tempest-VolumesTransfersV355Test-1097059843 None] Created volume successfully. Sep 17 21:15:35.724101 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] Acquiring lock "cinder-7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:15:35.745917 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] Lock "cinder-7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:15:35.821313 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:35.909859 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:15:35.910321 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] Volume volume-7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:15:35.930226 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] deleting rbd volume volume-7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:15:36.401434 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=52,cinder:UPDATE=39,cinder:INSERT=7,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:36.474302 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] Created reservations ['c55218ee-287f-46ea-8a9c-280bc8d0494d', '259e489a-bcc6-4569-9ab8-cee6ac853017', '005d3de1-7dcd-4c18-a451-e89efc594c23', 'd75d1d81-574f-407e-a968-5603f33e6ace'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:15:36.641872 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=58,cinder:UPDATE=31,cinder:INSERT=11 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:36.671858 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:36.782614 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:36.898955 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:15:36.905318 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] Deleted volume successfully. Sep 17 21:15:36.935995 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-da126ac3-801d-49f0-a378-e24412aabe63 tempest-VolumesTransfersV355Test-287664992 None] Lock "cinder-7cee3e6c-2a3f-4102-ae98-717ec6b5f6b7-delete_volume" released by "delete_volume" :: held 1.197s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:15:46.916257 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=7,cinder:SELECT=5,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:15:49.036569 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:15:49.080819 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Flow 'volume_create_manager' (428aedcf-2268-4833-a5a6-d73e4edd4f49) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:49.097190 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a050fa43-fe21-4a20-aef4-5c83c31c32da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:49.130318 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a050fa43-fe21-4a20-aef4-5c83c31c32da) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:15:49Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneNegativeTest-Volume-1277204313',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=973c99d9-9eef-4825-9cfa-9aa175ab3ff1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5b03de630e9147ea956fc60c721ac029',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:15:49Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:15:49Z,use_quota=True,user_id='af02475a88b6401d97b31b7c5563fe99',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:49.140572 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9fcc10a9-7931-44f3-a3a8-7fd71f554a6e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:49.153909 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9fcc10a9-7931-44f3-a3a8-7fd71f554a6e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:49.166655 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (31a039bc-7be0-4471-8979-a40b848535c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:49.174820 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (31a039bc-7be0-4471-8979-a40b848535c3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '973c99d9-9eef-4825-9cfa-9aa175ab3ff1', 'volume_name': 'volume-973c99d9-9eef-4825-9cfa-9aa175ab3ff1', 'volume_size': 2}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:49.182546 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (70b6d726-5221-475d-95ff-2a0999da5dbb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:49.197938 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (70b6d726-5221-475d-95ff-2a0999da5dbb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:49.209586 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cf47a6f5-507a-484e-8f45-f40596ddd640) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:49.211518 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Volume 973c99d9-9eef-4825-9cfa-9aa175ab3ff1: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-973c99d9-9eef-4825-9cfa-9aa175ab3ff1', 'volume_size': 2} Sep 17 21:15:49.214477 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] creating volume 'volume-973c99d9-9eef-4825-9cfa-9aa175ab3ff1' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:15:49.215459 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:49.333635 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:15:49.334051 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:15:49.338008 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cf47a6f5-507a-484e-8f45-f40596ddd640) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-973c99d9-9eef-4825-9cfa-9aa175ab3ff1', 'volume_size': 2}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:49.345521 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9a723679-da67-4e02-b2d8-38aad4340e68) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:49.364722 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Volume volume-973c99d9-9eef-4825-9cfa-9aa175ab3ff1 (973c99d9-9eef-4825-9cfa-9aa175ab3ff1): created successfully Sep 17 21:15:49.370046 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9a723679-da67-4e02-b2d8-38aad4340e68) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:49.378177 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Flow 'volume_create_manager' (428aedcf-2268-4833-a5a6-d73e4edd4f49) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:49.392769 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-1bb2ac0a-d638-401b-8756-0a57dc5ea80a tempest-VolumesCloneNegativeTest-700862021 None] Created volume successfully. Sep 17 21:15:50.359230 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] Acquiring lock "cinder-973c99d9-9eef-4825-9cfa-9aa175ab3ff1-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:15:50.377275 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] Lock "cinder-973c99d9-9eef-4825-9cfa-9aa175ab3ff1-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:15:50.452236 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:50.560279 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:15:50.560764 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] Volume volume-973c99d9-9eef-4825-9cfa-9aa175ab3ff1 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:15:50.574653 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] deleting rbd volume volume-973c99d9-9eef-4825-9cfa-9aa175ab3ff1 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:15:51.120685 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] Created reservations ['766f93b1-ab77-493f-91f7-eee0bd1a4bdc', '79e3c3d8-2741-4982-b8ea-d5275ca15037', '75cf1fec-0d39-4c93-bf63-a233d93c0512', '31304005-9d0e-4773-a867-784d524b5575'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:15:51.304684 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:51.414911 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:51.507333 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:15:51.522551 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] Deleted volume successfully. Sep 17 21:15:51.561290 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a3094d9e-5768-4cfa-a029-3aa3d16f2e1a tempest-VolumesCloneNegativeTest-700862021 None] Lock "cinder-973c99d9-9eef-4825-9cfa-9aa175ab3ff1-delete_volume" released by "delete_volume" :: held 1.184s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:15:56.388341 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:15:56.409558 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Flow 'volume_create_manager' (e9b8e47c-ddd7-4819-a271-61b089ff8ffb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:56.416228 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c3d00a75-4002-44cd-bcac-51471cef699e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:56.436216 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c3d00a75-4002-44cd-bcac-51471cef699e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:15:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeSnapshotQuotasNegativeTestJSON-Volume-1914951779',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1b4a5543-0018-4989-a7e7-c28ee0505a54,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='a98f75b3241841cdaa283fa4075d7593',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:15:56Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:15:56Z,use_quota=True,user_id='e494b95db6df4b9b8d119c6f70e2f81a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:56.443867 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6e8504ed-e54c-4fc4-91be-51d6a19a9c9c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:56.448932 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6e8504ed-e54c-4fc4-91be-51d6a19a9c9c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:56.461048 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (35b6ea21-103a-4dc7-8670-3bb1aab483b6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:56.465589 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (35b6ea21-103a-4dc7-8670-3bb1aab483b6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1b4a5543-0018-4989-a7e7-c28ee0505a54', 'volume_name': 'volume-1b4a5543-0018-4989-a7e7-c28ee0505a54', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:56.472531 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b6e23bb6-b704-4bfd-a9c6-c5b389edc031) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:56.479770 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b6e23bb6-b704-4bfd-a9c6-c5b389edc031) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:56.487435 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (312786f0-dedf-4007-8659-e6630b0e921b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:56.488532 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Volume 1b4a5543-0018-4989-a7e7-c28ee0505a54: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1b4a5543-0018-4989-a7e7-c28ee0505a54', 'volume_size': 1} Sep 17 21:15:56.488926 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] creating volume 'volume-1b4a5543-0018-4989-a7e7-c28ee0505a54' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:15:56.489733 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:56.645509 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:15:56.646165 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:15:56.650285 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (312786f0-dedf-4007-8659-e6630b0e921b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1b4a5543-0018-4989-a7e7-c28ee0505a54', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:56.659664 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (31095800-3d2b-41df-8f6b-356ab8cd2e95) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:15:56.698732 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Volume volume-1b4a5543-0018-4989-a7e7-c28ee0505a54 (1b4a5543-0018-4989-a7e7-c28ee0505a54): created successfully Sep 17 21:15:56.705651 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (31095800-3d2b-41df-8f6b-356ab8cd2e95) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:15:56.716753 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Flow 'volume_create_manager' (e9b8e47c-ddd7-4819-a271-61b089ff8ffb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:15:56.740830 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b19b8ab7-c536-4dd8-97cf-a1f5815f4ce6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Created volume successfully. Sep 17 21:15:57.782945 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-77da63ba-0894-4acd-b14c-7921aea353e6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:15:58.277379 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-77da63ba-0894-4acd-b14c-7921aea353e6 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Create snapshot completed successfully Sep 17 21:15:59.379337 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-490d82c4-d305-4985-9e68-8e2ac6366332 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Acquiring lock "cinder-9d84b925-8c77-4092-a01a-43720eeed348-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:15:59.402821 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-490d82c4-d305-4985-9e68-8e2ac6366332 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Lock "cinder-9d84b925-8c77-4092-a01a-43720eeed348-delete_snapshot" acquired by "delete_snapshot" :: waited 0.024s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:15:59.404605 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-490d82c4-d305-4985-9e68-8e2ac6366332 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:00.310642 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-490d82c4-d305-4985-9e68-8e2ac6366332 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Created reservations ['cfac9221-36a3-4862-af97-aeba153997a1', '70c38abe-9651-449c-8961-6d6e72176c21', '5f70a924-e738-46ee-80ac-78dbd582dc41', '2736e674-ab50-404f-a0d3-f65fb046c714'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:16:00.358987 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-490d82c4-d305-4985-9e68-8e2ac6366332 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Delete snapshot completed successfully. Sep 17 21:16:00.365533 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-490d82c4-d305-4985-9e68-8e2ac6366332 tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Lock "cinder-9d84b925-8c77-4092-a01a-43720eeed348-delete_snapshot" released by "delete_snapshot" :: held 0.963s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:16:00.589502 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Acquiring lock "cinder-1b4a5543-0018-4989-a7e7-c28ee0505a54-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:16:00.611775 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Lock "cinder-1b4a5543-0018-4989-a7e7-c28ee0505a54-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:16:00.682382 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:00.815977 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:16:00.816378 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Volume volume-1b4a5543-0018-4989-a7e7-c28ee0505a54 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:16:00.835791 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] deleting rbd volume volume-1b4a5543-0018-4989-a7e7-c28ee0505a54 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:16:01.032264 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:16:01.034220 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:01.123842 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:01.179719 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:16:01.324294 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Created reservations ['173c093a-8af7-4c90-a1b0-11334a3d7274', '25c3a11d-2655-488e-a8b6-7b3535c9c83b', 'b1d14aa4-48f9-4f8b-9e93-94553f2c2ed6', '046f6133-ffa2-4ab0-9e91-e17f2941069e'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:16:01.453207 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:01.569983 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:01.658229 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:16:01.670726 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Deleted volume successfully. Sep 17 21:16:01.698187 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-376df072-ccd9-4a05-972d-9f4dc605d47a tempest-VolumeSnapshotQuotasNegativeTestJSON-2015232249 None] Lock "cinder-1b4a5543-0018-4989-a7e7-c28ee0505a54-delete_volume" released by "delete_volume" :: held 1.089s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:16:08.815650 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-0726694d-04a0-4a2c-a98c-4244b2c672a7 req-2ca1b49d-dd04-44c5-b305-61530a6f8062 tempest-AttachVolumeShelveTestJSON-1971975439 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:16:10.837289 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-0726694d-04a0-4a2c-a98c-4244b2c672a7 req-2ca1b49d-dd04-44c5-b305-61530a6f8062 tempest-AttachVolumeShelveTestJSON-1971975439 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 2.021s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:16:10.838243 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-0726694d-04a0-4a2c-a98c-4244b2c672a7 req-2ca1b49d-dd04-44c5-b305-61530a6f8062 tempest-AttachVolumeShelveTestJSON-1971975439 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-b2bc9cfa-f40b-411e-8a44-20b3f6134615', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'b2bc9cfa-f40b-411e-8a44-20b3f6134615', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:16:10.853984 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=60,cinder:UPDATE=29,cinder:INSERT=3,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:10.888580 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-0726694d-04a0-4a2c-a98c-4244b2c672a7 req-2ca1b49d-dd04-44c5-b305-61530a6f8062 tempest-AttachVolumeShelveTestJSON-1971975439 None] Connection info returned from driver {'name': 'volumes/volume-b2bc9cfa-f40b-411e-8a44-20b3f6134615', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'b2bc9cfa-f40b-411e-8a44-20b3f6134615', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '3f05074d-3a19-43b2-ad51-7f42be269177'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:16:11.020733 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=60,cinder:UPDATE=28,cinder:DELETE=3,cinder:INSERT=9 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:11.021958 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-0726694d-04a0-4a2c-a98c-4244b2c672a7 req-2ca1b49d-dd04-44c5-b305-61530a6f8062 tempest-AttachVolumeShelveTestJSON-1971975439 None] attachment_update completed successfully. Sep 17 21:16:17.504214 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:16:17.535426 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Flow 'volume_create_manager' (32252f58-f2ce-4e1b-b1fe-3d11fe22c2c4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:16:17.542671 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (46e6c350-0358-4417-9247-246ae4f6c70f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:17.561413 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (46e6c350-0358-4417-9247-246ae4f6c70f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:16:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-2088730390',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a67ccfd5-d481-4007-b0ed-820dffc9082c,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='3fe15d4415344341b9d9d8f7694cddc3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:16:17Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:16:17Z,use_quota=True,user_id='262593ad1bf34506a4d7f73bf682c7d1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:17.567421 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4a3707d5-eeb8-42e5-8f18-f9ba909bb47b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:17.572889 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4a3707d5-eeb8-42e5-8f18-f9ba909bb47b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:17.578531 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e7a63249-c3a7-4089-a29a-3efded2171f1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:17.583198 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e7a63249-c3a7-4089-a29a-3efded2171f1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a67ccfd5-d481-4007-b0ed-820dffc9082c', 'volume_name': 'volume-a67ccfd5-d481-4007-b0ed-820dffc9082c', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:17.588635 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (61a56744-5b8e-41e1-a3be-87b622bf594a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:17.593531 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (61a56744-5b8e-41e1-a3be-87b622bf594a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:17.600155 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a06deec8-92a7-460f-9db3-6d18e5b741f0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:17.601988 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Volume a67ccfd5-d481-4007-b0ed-820dffc9082c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a67ccfd5-d481-4007-b0ed-820dffc9082c', 'volume_size': 1} Sep 17 21:16:17.602865 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] creating volume 'volume-a67ccfd5-d481-4007-b0ed-820dffc9082c' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:16:17.603926 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:17.773188 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:16:17.773603 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:16:17.777570 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a06deec8-92a7-460f-9db3-6d18e5b741f0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a67ccfd5-d481-4007-b0ed-820dffc9082c', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:17.784659 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5d098039-9251-4f9f-8731-3c2cbef34504) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:17.805415 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Volume volume-a67ccfd5-d481-4007-b0ed-820dffc9082c (a67ccfd5-d481-4007-b0ed-820dffc9082c): created successfully Sep 17 21:16:17.812168 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5d098039-9251-4f9f-8731-3c2cbef34504) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:17.828010 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Flow 'volume_create_manager' (32252f58-f2ce-4e1b-b1fe-3d11fe22c2c4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:16:17.871297 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a23476bb-6df3-4c9e-9d44-5982fd65510d tempest-VolumesListTestJSON-1070833108 None] Created volume successfully. Sep 17 21:16:19.275259 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:16:19.335360 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Flow 'volume_create_manager' (4de834b7-2aea-4861-b529-b4e3eeb406dc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:16:19.350748 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a07e80c6-6014-42b4-8608-99e248c2b612) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:19.391148 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a07e80c6-6014-42b4-8608-99e248c2b612) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:16:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-237879153',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8f4db2d9-a565-4bd5-bf2b-78d48eb13085,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='3fe15d4415344341b9d9d8f7694cddc3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:16:19Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:16:19Z,use_quota=True,user_id='262593ad1bf34506a4d7f73bf682c7d1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:19.402899 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1b28db8f-6792-4866-af1c-8b9707c99e49) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:19.420509 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1b28db8f-6792-4866-af1c-8b9707c99e49) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:19.426232 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ad722fb0-4158-4ef9-be50-e3f39223de17) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:19.441923 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ad722fb0-4158-4ef9-be50-e3f39223de17) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8f4db2d9-a565-4bd5-bf2b-78d48eb13085', 'volume_name': 'volume-8f4db2d9-a565-4bd5-bf2b-78d48eb13085', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:19.454024 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (71619b0d-bfb0-49ec-953c-92c4406696f0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:19.458409 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (71619b0d-bfb0-49ec-953c-92c4406696f0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:19.468884 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (28348b31-4a2e-4a30-b0b3-bfafeb1a368a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:19.470039 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Volume 8f4db2d9-a565-4bd5-bf2b-78d48eb13085: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8f4db2d9-a565-4bd5-bf2b-78d48eb13085', 'volume_size': 1} Sep 17 21:16:19.476760 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] creating volume 'volume-8f4db2d9-a565-4bd5-bf2b-78d48eb13085' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:16:19.477759 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:19.663900 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:16:19.664762 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:16:19.668827 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (28348b31-4a2e-4a30-b0b3-bfafeb1a368a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8f4db2d9-a565-4bd5-bf2b-78d48eb13085', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:19.677818 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (49c921d0-f50d-4ad0-ac8d-da5cc5a2bdc3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:19.737557 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Volume volume-8f4db2d9-a565-4bd5-bf2b-78d48eb13085 (8f4db2d9-a565-4bd5-bf2b-78d48eb13085): created successfully Sep 17 21:16:19.755056 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (49c921d0-f50d-4ad0-ac8d-da5cc5a2bdc3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:19.768797 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Flow 'volume_create_manager' (4de834b7-2aea-4861-b529-b4e3eeb406dc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:16:19.807759 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-0fb3c06f-fa92-4c10-96f0-05fdb961336a tempest-VolumesListTestJSON-1070833108 None] Created volume successfully. Sep 17 21:16:21.417206 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:16:21.490660 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Flow 'volume_create_manager' (6989a225-225b-429e-81f4-e4841a9152ac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:16:21.507476 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9169b071-c651-41c6-a19c-2503c0577045) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:21.560886 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9169b071-c651-41c6-a19c-2503c0577045) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:16:21Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-2132336968',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=be3de1ac-34eb-4061-83aa-2ce684e1da4b,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='3fe15d4415344341b9d9d8f7694cddc3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:16:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:16:21Z,use_quota=True,user_id='262593ad1bf34506a4d7f73bf682c7d1',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:21.571686 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (62d76573-2ff1-4145-afd1-a4e708a3f35b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:21.583986 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (62d76573-2ff1-4145-afd1-a4e708a3f35b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:21.602247 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (58b6ade2-b42f-460f-9437-fa74c3cc66be) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:21.606489 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (58b6ade2-b42f-460f-9437-fa74c3cc66be) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'be3de1ac-34eb-4061-83aa-2ce684e1da4b', 'volume_name': 'volume-be3de1ac-34eb-4061-83aa-2ce684e1da4b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:21.612227 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (08ed5904-aae0-4058-8a20-47fe326aaccf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:21.616056 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (08ed5904-aae0-4058-8a20-47fe326aaccf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:21.628709 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9f9d68a6-a8b8-4741-80fd-a90a16cdd20a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:21.629838 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Volume be3de1ac-34eb-4061-83aa-2ce684e1da4b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-be3de1ac-34eb-4061-83aa-2ce684e1da4b', 'volume_size': 1} Sep 17 21:16:21.630224 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] creating volume 'volume-be3de1ac-34eb-4061-83aa-2ce684e1da4b' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:16:21.630877 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:21.792477 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:16:21.793336 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:16:21.802916 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9f9d68a6-a8b8-4741-80fd-a90a16cdd20a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-be3de1ac-34eb-4061-83aa-2ce684e1da4b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:21.822887 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0b51ffaf-4036-41b6-b4c0-ca7fb657163e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:16:21.853545 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Volume volume-be3de1ac-34eb-4061-83aa-2ce684e1da4b (be3de1ac-34eb-4061-83aa-2ce684e1da4b): created successfully Sep 17 21:16:21.862196 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0b51ffaf-4036-41b6-b4c0-ca7fb657163e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:16:21.880452 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Flow 'volume_create_manager' (6989a225-225b-429e-81f4-e4841a9152ac) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:16:21.903806 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7d3d1360-d14e-4d0c-8ab4-13037764e58d tempest-VolumesListTestJSON-1070833108 None] Created volume successfully. Sep 17 21:16:29.206180 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] Acquiring lock "cinder-be3de1ac-34eb-4061-83aa-2ce684e1da4b-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:16:29.228127 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] Lock "cinder-be3de1ac-34eb-4061-83aa-2ce684e1da4b-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:16:29.341215 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:29.459732 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:16:29.460419 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] Volume volume-be3de1ac-34eb-4061-83aa-2ce684e1da4b is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:16:29.472025 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] deleting rbd volume volume-be3de1ac-34eb-4061-83aa-2ce684e1da4b {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:16:29.889297 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] Created reservations ['bf2fcabe-81d4-4ef1-92ad-8f714e529b93', '14c6af70-fdbe-4d5e-b1be-2a2bf194a355', 'a2de5bbf-30a0-4094-99c9-ed02546a0705', 'caec9427-6192-455d-9e11-3ec8826d2453'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:16:30.056374 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:30.115804 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:30.199920 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:16:30.204786 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] Deleted volume successfully. Sep 17 21:16:30.227534 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ace20219-e201-4fd5-b6e4-3a60579e6c53 tempest-VolumesListTestJSON-1070833108 None] Lock "cinder-be3de1ac-34eb-4061-83aa-2ce684e1da4b-delete_volume" released by "delete_volume" :: held 1.000s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:16:30.566472 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] Acquiring lock "cinder-8f4db2d9-a565-4bd5-bf2b-78d48eb13085-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:16:30.591752 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] Lock "cinder-8f4db2d9-a565-4bd5-bf2b-78d48eb13085-delete_volume" acquired by "delete_volume" :: waited 0.028s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:16:30.712028 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:30.868922 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:16:30.869317 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] Volume volume-8f4db2d9-a565-4bd5-bf2b-78d48eb13085 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:16:30.883154 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] deleting rbd volume volume-8f4db2d9-a565-4bd5-bf2b-78d48eb13085 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:16:31.286022 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] Created reservations ['f7cf5483-a993-4646-8c1a-e63ea99f8c37', '105f4761-7ab7-4d1f-be14-671d97dded1c', '0ea4d344-1748-4e4d-bf9c-055d785651ea', 'e8ed8c35-e96d-49a4-b914-c510fe94e635'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:16:31.400417 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:31.509024 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:31.618468 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:16:31.628531 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] Deleted volume successfully. Sep 17 21:16:31.678534 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-c2e683f0-613e-4a9a-9eba-ecb2c0360011 tempest-VolumesListTestJSON-1070833108 None] Lock "cinder-8f4db2d9-a565-4bd5-bf2b-78d48eb13085-delete_volume" released by "delete_volume" :: held 1.087s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:16:31.903878 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] Acquiring lock "cinder-a67ccfd5-d481-4007-b0ed-820dffc9082c-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:16:31.932805 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] Lock "cinder-a67ccfd5-d481-4007-b0ed-820dffc9082c-delete_volume" acquired by "delete_volume" :: waited 0.029s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:16:32.128761 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:32.257395 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:16:32.258194 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] Volume volume-a67ccfd5-d481-4007-b0ed-820dffc9082c is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:16:32.274425 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] deleting rbd volume volume-a67ccfd5-d481-4007-b0ed-820dffc9082c {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:16:32.801196 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] Created reservations ['71a66158-f3c2-47e4-a1f3-68af1fb16b1f', '737bcfe2-72fd-4787-a188-91e0affb28b7', 'a44ee9b0-42ab-45ac-969d-7400dc2a5bdc', '82d9bab2-a08e-47c8-84ab-ce00cfc938d0'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:16:32.962997 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:33.056630 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:16:33.127719 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:16:33.132222 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] Deleted volume successfully. Sep 17 21:16:33.164730 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3b9efdea-b4f4-4819-b8bc-27e4be963ecc tempest-VolumesListTestJSON-1070833108 None] Lock "cinder-a67ccfd5-d481-4007-b0ed-820dffc9082c-delete_volume" released by "delete_volume" :: held 1.232s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:16:42.710914 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=52,cinder:UPDATE=28,cinder:DELETE=5,cinder:INSERT=8 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:16:42.712195 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=56,cinder:UPDATE=32,cinder:INSERT=4,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:01.035755 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:17:01.036454 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:17:01.158073 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:17:01.249008 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:17:02.719022 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:16.893789 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=10,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:16.919435 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:17:16.951202 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Flow 'volume_create_manager' (3f182574-bb41-488c-a364-e94329c42bfe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:17:16.961705 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4a51fda1-bd67-448d-8693-2590ad3a053b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:17:16.988010 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4a51fda1-bd67-448d-8693-2590ad3a053b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:17:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-485816957',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=779942d3-ce89-44ed-bbfc-e00719495d8e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8d3fab3adf0c425680c91af8afcc516f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:17:17Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:17:17Z,use_quota=True,user_id='7ff5b886208a44c0891c2d1ba65dc083',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:17:16.995966 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (98eaef40-2da1-46fa-b027-e386733b5bf9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:17:17.003254 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (98eaef40-2da1-46fa-b027-e386733b5bf9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:17:17.010110 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0d46fe71-0242-47bd-ba11-1baee11e1c8f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:17:17.015418 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0d46fe71-0242-47bd-ba11-1baee11e1c8f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '779942d3-ce89-44ed-bbfc-e00719495d8e', 'volume_name': 'volume-779942d3-ce89-44ed-bbfc-e00719495d8e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:17:17.023152 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a66ee914-424d-46e8-9fe6-3f811eb08b52) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:17:17.027537 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a66ee914-424d-46e8-9fe6-3f811eb08b52) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:17:17.032591 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8ce6e79d-344d-4fe5-bc90-716d4d01f3d9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:17:17.033895 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Volume 779942d3-ce89-44ed-bbfc-e00719495d8e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-779942d3-ce89-44ed-bbfc-e00719495d8e', 'volume_size': 1} Sep 17 21:17:17.034439 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] creating volume 'volume-779942d3-ce89-44ed-bbfc-e00719495d8e' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:17:17.035538 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:17:17.154122 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:17:17.156559 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:17:17.163231 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8ce6e79d-344d-4fe5-bc90-716d4d01f3d9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-779942d3-ce89-44ed-bbfc-e00719495d8e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:17:17.170936 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4f670f13-89b7-439a-bea9-a60306aaa412) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:17:17.184796 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Volume volume-779942d3-ce89-44ed-bbfc-e00719495d8e (779942d3-ce89-44ed-bbfc-e00719495d8e): created successfully Sep 17 21:17:17.192726 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4f670f13-89b7-439a-bea9-a60306aaa412) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:17:17.206301 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Flow 'volume_create_manager' (3f182574-bb41-488c-a364-e94329c42bfe) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:17:17.225710 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-63531671-624a-4594-b37c-d5c68e87ce28 tempest-AttachVolumeTestJSON-890686982 None] Created volume successfully. Sep 17 21:17:18.632225 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-342fcc2c-02a4-4830-ba42-689531c67116 req-59f12f72-0e9f-4eb7-8cf2-3be12a84d0d2 tempest-AttachVolumeTestJSON-890686982 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:17:19.686100 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-8a0b749e-4abd-4d21-9f8f-6780dea2e55e req-436d52b4-8c18-46cd-b030-5d99cf2d8013 tempest-AttachVolumeShelveTestJSON-1971975439 None] Terminate volume connection completed successfully. Sep 17 21:17:19.687379 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-8a0b749e-4abd-4d21-9f8f-6780dea2e55e req-436d52b4-8c18-46cd-b030-5d99cf2d8013 tempest-AttachVolumeShelveTestJSON-1971975439 None] Deleting attachment 3f05074d-3a19-43b2-ad51-7f42be269177. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:17:19.798096 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-342fcc2c-02a4-4830-ba42-689531c67116 req-59f12f72-0e9f-4eb7-8cf2-3be12a84d0d2 tempest-AttachVolumeTestJSON-890686982 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.166s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:17:19.798791 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-342fcc2c-02a4-4830-ba42-689531c67116 req-59f12f72-0e9f-4eb7-8cf2-3be12a84d0d2 tempest-AttachVolumeTestJSON-890686982 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-779942d3-ce89-44ed-bbfc-e00719495d8e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '779942d3-ce89-44ed-bbfc-e00719495d8e', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:17:19.848706 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-342fcc2c-02a4-4830-ba42-689531c67116 req-59f12f72-0e9f-4eb7-8cf2-3be12a84d0d2 tempest-AttachVolumeTestJSON-890686982 None] Connection info returned from driver {'name': 'volumes/volume-779942d3-ce89-44ed-bbfc-e00719495d8e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '779942d3-ce89-44ed-bbfc-e00719495d8e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'd03fa9e1-252b-4392-97e2-cace6241f039'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:17:19.920447 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-342fcc2c-02a4-4830-ba42-689531c67116 req-59f12f72-0e9f-4eb7-8cf2-3be12a84d0d2 tempest-AttachVolumeTestJSON-890686982 None] attachment_update completed successfully. Sep 17 21:17:29.919768 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=17,cinder:UPDATE=4,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:32.721326 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=18,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:42.723229 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:17:52.741529 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:01.037637 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:18:01.037637 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:01.076362 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:01.112671 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:18:04.432393 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:18:04.491032 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Flow 'volume_create_manager' (27f6bd62-a96e-4506-a62c-548b96335270) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:04.499390 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a53772c0-eb86-4359-a99a-223ffb55e5ad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:04.521904 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a53772c0-eb86-4359-a99a-223ffb55e5ad) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:18:04Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasAdminTestJSON-Volume-1432398172',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=46dabf05-ca41-42ed-b1cc-39bd2309ea2d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c81aa9c09de54cbb9103d6b8bc6442ad',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:18:04Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:18:04Z,use_quota=True,user_id='28cd2509195a4093b08e9b4dbdd8e931',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:04.540177 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ac7f2e29-50a9-402e-ae86-53cb506a7475) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:04.551033 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ac7f2e29-50a9-402e-ae86-53cb506a7475) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:04.566796 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ecdfa61a-1607-4e18-a098-b546893aac6f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:04.582769 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ecdfa61a-1607-4e18-a098-b546893aac6f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '46dabf05-ca41-42ed-b1cc-39bd2309ea2d', 'volume_name': 'volume-46dabf05-ca41-42ed-b1cc-39bd2309ea2d', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:04.594921 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4ca810f1-110d-4cdd-af28-c0c38f93c74a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:04.599731 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4ca810f1-110d-4cdd-af28-c0c38f93c74a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:04.605016 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (63be3bc1-7b73-4a12-abb7-a8138010aa24) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:04.606641 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Volume 46dabf05-ca41-42ed-b1cc-39bd2309ea2d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-46dabf05-ca41-42ed-b1cc-39bd2309ea2d', 'volume_size': 1} Sep 17 21:18:04.607196 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] creating volume 'volume-46dabf05-ca41-42ed-b1cc-39bd2309ea2d' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:18:04.608004 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:04.716900 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:04.717328 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:04.721258 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (63be3bc1-7b73-4a12-abb7-a8138010aa24) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-46dabf05-ca41-42ed-b1cc-39bd2309ea2d', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:04.725810 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6b61dab3-e9ee-4953-a393-0842974010d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:04.739444 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Volume volume-46dabf05-ca41-42ed-b1cc-39bd2309ea2d (46dabf05-ca41-42ed-b1cc-39bd2309ea2d): created successfully Sep 17 21:18:04.749817 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6b61dab3-e9ee-4953-a393-0842974010d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:04.758920 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Flow 'volume_create_manager' (27f6bd62-a96e-4506-a62c-548b96335270) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:04.775024 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7c1b899a-a1b3-4a11-a70b-f93810fa6027 tempest-VolumeQuotasAdminTestJSON-711835996 None] Created volume successfully. Sep 17 21:18:05.557318 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] Acquiring lock "cinder-46dabf05-ca41-42ed-b1cc-39bd2309ea2d-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:18:05.577784 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] Lock "cinder-46dabf05-ca41-42ed-b1cc-39bd2309ea2d-delete_volume" acquired by "delete_volume" :: waited 0.020s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:18:05.668108 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:05.762428 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:18:05.763089 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] Volume volume-46dabf05-ca41-42ed-b1cc-39bd2309ea2d is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:18:05.771154 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] deleting rbd volume volume-46dabf05-ca41-42ed-b1cc-39bd2309ea2d {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:18:06.210783 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] Created reservations ['dab235a6-c834-45d0-aaf0-b26dfe26b7a7', '5c51dcff-dcfb-41d7-b5c1-f823968f4f8e', '4d6e0915-2f57-4d6c-9eee-4fe8671f8f27', '134ae064-4fc2-41c5-b747-74266bfc17f1'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:06.316270 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:06.432117 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:06.503254 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:18:06.509258 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] Deleted volume successfully. Sep 17 21:18:06.544487 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-0db68b44-9a74-4124-a26d-be896f79f9b0 tempest-VolumeQuotasAdminTestJSON-711835996 None] Lock "cinder-46dabf05-ca41-42ed-b1cc-39bd2309ea2d-delete_volume" released by "delete_volume" :: held 0.964s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:18:07.431499 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:18:07.470239 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Flow 'volume_create_manager' (832f63c6-c5c4-4a3e-84dc-67d0f0e8125e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:07.477578 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3ddf0d53-0002-4a92-ab3b-a100f6e6cc43) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:07.496556 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3ddf0d53-0002-4a92-ab3b-a100f6e6cc43) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:18:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasAdminTestJSON-Volume-598695637',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=568660cb-5eee-43b0-bd39-fcaf2bc3aee8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c81aa9c09de54cbb9103d6b8bc6442ad',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:18:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:18:07Z,use_quota=True,user_id='28cd2509195a4093b08e9b4dbdd8e931',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:07.502908 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6e0d5265-569c-45bf-aca8-add0b439d8e3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:07.507489 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6e0d5265-569c-45bf-aca8-add0b439d8e3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:07.512555 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b2e7e3db-a4a7-4d9f-a5ab-976b7968f957) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:07.517378 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b2e7e3db-a4a7-4d9f-a5ab-976b7968f957) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '568660cb-5eee-43b0-bd39-fcaf2bc3aee8', 'volume_name': 'volume-568660cb-5eee-43b0-bd39-fcaf2bc3aee8', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:07.523158 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6f21c321-c532-4e5a-b7da-af791311deec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:07.527703 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6f21c321-c532-4e5a-b7da-af791311deec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:07.532299 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5425cacf-a647-46ff-bbf7-b55de3b24aee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:07.533965 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Volume 568660cb-5eee-43b0-bd39-fcaf2bc3aee8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-568660cb-5eee-43b0-bd39-fcaf2bc3aee8', 'volume_size': 1} Sep 17 21:18:07.534508 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] creating volume 'volume-568660cb-5eee-43b0-bd39-fcaf2bc3aee8' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:18:07.535428 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:07.612405 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:07.613029 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:07.617268 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5425cacf-a647-46ff-bbf7-b55de3b24aee) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-568660cb-5eee-43b0-bd39-fcaf2bc3aee8', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:07.623706 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (87b20c1f-8fe8-4508-a5b2-d84fc9f1316d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:07.637743 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Volume volume-568660cb-5eee-43b0-bd39-fcaf2bc3aee8 (568660cb-5eee-43b0-bd39-fcaf2bc3aee8): created successfully Sep 17 21:18:07.642838 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (87b20c1f-8fe8-4508-a5b2-d84fc9f1316d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:07.650954 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Flow 'volume_create_manager' (832f63c6-c5c4-4a3e-84dc-67d0f0e8125e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:07.670556 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f71b5103-e090-4655-8da0-b416e123f481 tempest-VolumeQuotasAdminTestJSON-711835996 None] Created volume successfully. Sep 17 21:18:09.400828 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2053ad08-a95f-4da6-9e31-daf541edfaa9 tempest-VolumeQuotasAdminTestJSON-474905647 None] Transfer volume completed successfully. Sep 17 21:18:09.848698 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] Acquiring lock "cinder-568660cb-5eee-43b0-bd39-fcaf2bc3aee8-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:18:09.871058 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] Lock "cinder-568660cb-5eee-43b0-bd39-fcaf2bc3aee8-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:18:09.920999 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:10.007693 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:18:10.008133 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] Volume volume-568660cb-5eee-43b0-bd39-fcaf2bc3aee8 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:18:10.018275 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] deleting rbd volume volume-568660cb-5eee-43b0-bd39-fcaf2bc3aee8 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:18:10.356343 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] Created reservations ['0a8fdaba-3f75-47bb-b2e0-e9267f52ac61', '83e2b335-0273-4dc7-9474-e2748d669f20', '4bf97617-fde0-49e7-bafd-cd7d595602c7', '9deaae5d-321c-4caf-867d-32ce4f4ac0bf'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:10.423526 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:10.480834 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:10.573857 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:18:10.584950 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] Deleted volume successfully. Sep 17 21:18:10.601389 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f86d5ac1-4cf8-4bbb-9b10-a28db2ed146d tempest-VolumeQuotasAdminTestJSON-833552219 None] Lock "cinder-568660cb-5eee-43b0-bd39-fcaf2bc3aee8-delete_volume" released by "delete_volume" :: held 0.730s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:18:12.143733 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-00ece85b-0319-4874-8d20-2ca7af1fee1b req-de47d209-5422-42d8-acef-669289af8b56 tempest-AttachVolumeTestJSON-890686982 None] Terminate volume connection completed successfully. Sep 17 21:18:12.144965 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-00ece85b-0319-4874-8d20-2ca7af1fee1b req-de47d209-5422-42d8-acef-669289af8b56 tempest-AttachVolumeTestJSON-890686982 None] Deleting attachment d03fa9e1-252b-4392-97e2-cace6241f039. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:18:14.671315 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=35,cinder:UPDATE=29,cinder:INSERT=2,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:14.683426 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-516ed378-fd04-4ab0-a2a7-35a3d718b835 tempest-GroupSnapshotsTest-330488771 None] Group tempest-GroupSnapshotsTest-Group-3320564: creating Sep 17 21:18:14.704740 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-516ed378-fd04-4ab0-a2a7-35a3d718b835 tempest-GroupSnapshotsTest-330488771 None] Group tempest-GroupSnapshotsTest-Group-3320564: created successfully Sep 17 21:18:14.715065 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-516ed378-fd04-4ab0-a2a7-35a3d718b835 tempest-GroupSnapshotsTest-330488771 None] Create group completed successfully. Sep 17 21:18:15.964286 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:18:15.982946 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Flow 'volume_create_manager' (b50f3d15-7aa7-4db2-9776-4fa43e153a11) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:15.989071 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (81202661-6f90-4f04-9dc7-0da87da79aa0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.004994 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (81202661-6f90-4f04-9dc7-0da87da79aa0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:18:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeRetypeWithoutMigrationTest-Volume-46823449',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a5e67838-46f0-45d6-b9fb-48c4c0edd77e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='111884222b25460a9613135036eb9c9f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:18:16Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:18:16Z,use_quota=True,user_id='7fad04528a2d48698acb72682d1a2b86',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(ed35c46f-25a7-4aca-8e14-2a7837eb1456),volume_type_id=ed35c46f-25a7-4aca-8e14-2a7837eb1456)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.011375 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (80532c70-1311-49ae-ba0a-90f5a51c2acc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.015701 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (80532c70-1311-49ae-ba0a-90f5a51c2acc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.020865 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e08640c2-f569-4160-a2ea-4e8733e1eaf8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.025081 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e08640c2-f569-4160-a2ea-4e8733e1eaf8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a5e67838-46f0-45d6-b9fb-48c4c0edd77e', 'volume_name': 'volume-a5e67838-46f0-45d6-b9fb-48c4c0edd77e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.029809 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac9bfb98-0b10-4c79-a7c6-e204c5243940) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.033928 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac9bfb98-0b10-4c79-a7c6-e204c5243940) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.038443 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0891718f-1c3d-4ec8-8afd-ce1e34e41db1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.039503 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Volume a5e67838-46f0-45d6-b9fb-48c4c0edd77e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a5e67838-46f0-45d6-b9fb-48c4c0edd77e', 'volume_size': 1} Sep 17 21:18:16.040034 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] creating volume 'volume-a5e67838-46f0-45d6-b9fb-48c4c0edd77e' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:18:16.041001 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:16.107425 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:16.107877 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:16.111702 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0891718f-1c3d-4ec8-8afd-ce1e34e41db1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a5e67838-46f0-45d6-b9fb-48c4c0edd77e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.116485 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (32a28b2d-cd72-4c56-960a-d204f8a54214) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.136636 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Volume volume-a5e67838-46f0-45d6-b9fb-48c4c0edd77e (a5e67838-46f0-45d6-b9fb-48c4c0edd77e): created successfully Sep 17 21:18:16.140259 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (32a28b2d-cd72-4c56-960a-d204f8a54214) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.145719 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Flow 'volume_create_manager' (b50f3d15-7aa7-4db2-9776-4fa43e153a11) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:16.155531 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c89519b5-d055-40c2-989b-dd94be5db681 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Created volume successfully. Sep 17 21:18:16.161274 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:18:16.179961 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Flow 'volume_create_manager' (535b76c2-e31c-4f52-83d7-4c60aaf1b37c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:16.186172 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f73d31fd-03f9-4edb-bc01-cd46c6867c50) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.201507 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f73d31fd-03f9-4edb-bc01-cd46c6867c50) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:18:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-349014924',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=f11a8785-7d63-4b84-ae15-c0239fb9524e,host='ncdv-master-1@ceph#ceph',id=ceb0b14c-98ee-4885-aa9c-e63a03575bab,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f5457492c1db4888802770dcf7055c1d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:18:16Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:18:16Z,use_quota=True,user_id='df2f60fa0bde4397aaabf6caa8e4efe2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(55eaf579-8f65-40cb-a57d-b95f6c3349ce),volume_type_id=55eaf579-8f65-40cb-a57d-b95f6c3349ce)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.206718 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (286cb8f8-45d3-4d0d-9261-f1250ed624ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.210943 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (286cb8f8-45d3-4d0d-9261-f1250ed624ff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.215841 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (aed8b639-cb30-450d-8d6f-f1134d131ca3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.219795 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (aed8b639-cb30-450d-8d6f-f1134d131ca3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ceb0b14c-98ee-4885-aa9c-e63a03575bab', 'volume_name': 'volume-ceb0b14c-98ee-4885-aa9c-e63a03575bab', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.224549 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (276e9fda-bbd5-44e9-ab8c-70fc57d6ab94) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.228462 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (276e9fda-bbd5-44e9-ab8c-70fc57d6ab94) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.233023 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (647cd107-9829-47c6-960d-1c4a316fb98e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.234161 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Volume ceb0b14c-98ee-4885-aa9c-e63a03575bab: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ceb0b14c-98ee-4885-aa9c-e63a03575bab', 'volume_size': 1} Sep 17 21:18:16.234768 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] creating volume 'volume-ceb0b14c-98ee-4885-aa9c-e63a03575bab' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:18:16.235195 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:16.289841 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:16.290259 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:16.294244 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (647cd107-9829-47c6-960d-1c4a316fb98e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ceb0b14c-98ee-4885-aa9c-e63a03575bab', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.298818 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a7b47361-6eb8-4145-9777-8acd30febb44) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:16.309669 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Volume volume-ceb0b14c-98ee-4885-aa9c-e63a03575bab (ceb0b14c-98ee-4885-aa9c-e63a03575bab): created successfully Sep 17 21:18:16.313559 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a7b47361-6eb8-4145-9777-8acd30febb44) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:16.319158 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Flow 'volume_create_manager' (535b76c2-e31c-4f52-83d7-4c60aaf1b37c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:16.329030 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e12c378c-57b3-441a-a070-f3c5e5ceba2c tempest-GroupSnapshotsTest-330488771 None] Created volume successfully. Sep 17 21:18:17.317568 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7c1c6e48-695b-4855-b011-960d56e5a566 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:17.345716 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7c1c6e48-695b-4855-b011-960d56e5a566 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:17.376820 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-7c1c6e48-695b-4855-b011-960d56e5a566 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:18:17.381689 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7c1c6e48-695b-4855-b011-960d56e5a566 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Retype volume completed successfully. Sep 17 21:18:17.392279 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6da41e74-2c4b-4250-95b3-a5d3c40f6a1c tempest-GroupSnapshotsTest-330488771 None] GroupSnapshot 47573027-31a0-4b3f-886b-bf0bdfcd8844: creating. Sep 17 21:18:17.425564 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6da41e74-2c4b-4250-95b3-a5d3c40f6a1c tempest-GroupSnapshotsTest-330488771 None] Group snapshot 47573027-31a0-4b3f-886b-bf0bdfcd8844: creating. {{(pid=133409) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Sep 17 21:18:17.447343 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6da41e74-2c4b-4250-95b3-a5d3c40f6a1c tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:18.294203 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=68,cinder:UPDATE=22,cinder:DELETE=4,cinder:INSERT=6 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:18.347005 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6da41e74-2c4b-4250-95b3-a5d3c40f6a1c tempest-GroupSnapshotsTest-330488771 None] group_snapshot 47573027-31a0-4b3f-886b-bf0bdfcd8844: created successfully Sep 17 21:18:18.545746 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Acquiring lock "cinder-a5e67838-46f0-45d6-b9fb-48c4c0edd77e-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:18:18.560446 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Lock "cinder-a5e67838-46f0-45d6-b9fb-48c4c0edd77e-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:18:18.602257 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:18.653390 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:18:18.653890 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Volume volume-a5e67838-46f0-45d6-b9fb-48c4c0edd77e is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:18:18.659350 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] deleting rbd volume volume-a5e67838-46f0-45d6-b9fb-48c4c0edd77e {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:18:18.861248 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Deprecated: Default quota for resource: volumes_volume-type-2 is set by the default quota flag: quota_volumes_volume-type-2, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:18:18.862093 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Deprecated: Default quota for resource: gigabytes_volume-type-2 is set by the default quota flag: quota_gigabytes_volume-type-2, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:18:18.877823 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Created reservations ['2b3bdd58-48fa-473e-9cf5-ba7f56b98c6d', 'f3d62d3e-1c18-430a-b78a-61d5c85a2bde', '06a98d16-5352-41d8-b0be-8208dd890a24', '377879c9-b9db-474d-8604-137f8c241ecb'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:18.923463 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:18.949935 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:18.978445 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:18:18.983228 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Deleted volume successfully. Sep 17 21:18:18.997412 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2cb0f151-145c-4698-9016-b7b9ff415f37 tempest-VolumeRetypeWithoutMigrationTest-207737403 None] Lock "cinder-a5e67838-46f0-45d6-b9fb-48c4c0edd77e-delete_volume" released by "delete_volume" :: held 0.437s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:18:19.056622 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9fde872-a61f-4fd0-b294-861ff1f1a009 tempest-GroupSnapshotsTest-330488771 None] cloning volumes/volume-ceb0b14c-98ee-4885-aa9c-e63a03575bab@snapshot-e0e9755a-8214-4c7b-badb-aebb703dba39 to volume-c54629ed-227c-47f9-ba1e-c1168ac2b053 {{(pid=133409) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 17 21:18:19.057273 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9fde872-a61f-4fd0-b294-861ff1f1a009 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:19.095362 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9fde872-a61f-4fd0-b294-861ff1f1a009 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:19.197793 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9fde872-a61f-4fd0-b294-861ff1f1a009 tempest-GroupSnapshotsTest-330488771 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:19.198195 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9fde872-a61f-4fd0-b294-861ff1f1a009 tempest-GroupSnapshotsTest-330488771 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:19.206396 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9fde872-a61f-4fd0-b294-861ff1f1a009 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:19.242357 ncdv-master-1 cinder-volume[133409]: WARNING cinder.volume.drivers.rbd [None req-e9fde872-a61f-4fd0-b294-861ff1f1a009 tempest-GroupSnapshotsTest-330488771 None] Not using v2 clone API, please upgrade to mimic+ and set the OSD minimum client compat version to mimic for better performance, fewer deletion issues: NotImplementedError Sep 17 21:18:19.312738 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e9fde872-a61f-4fd0-b294-861ff1f1a009 tempest-GroupSnapshotsTest-330488771 None] Create group from source-snapshot-47573027-31a0-4b3f-886b-bf0bdfcd8844 completed successfully. Sep 17 21:18:20.761042 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2ce1569-df1e-4db3-b434-56e9377aa881 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:20.858076 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2ce1569-df1e-4db3-b434-56e9377aa881 tempest-GroupSnapshotsTest-330488771 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:18:20.874287 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2ce1569-df1e-4db3-b434-56e9377aa881 tempest-GroupSnapshotsTest-330488771 None] deleting rbd volume volume-c54629ed-227c-47f9-ba1e-c1168ac2b053 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:18:21.121800 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-c2ce1569-df1e-4db3-b434-56e9377aa881 tempest-GroupSnapshotsTest-330488771 None] Created reservations ['1dc8e01b-4744-4647-87c9-b4a5b1bfb058'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:21.125134 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=75,cinder:UPDATE=21,cinder:INSERT=3,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:21.149056 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-c2ce1569-df1e-4db3-b434-56e9377aa881 tempest-GroupSnapshotsTest-330488771 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-1565505675 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-1565505675, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:18:21.149964 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-c2ce1569-df1e-4db3-b434-56e9377aa881 tempest-GroupSnapshotsTest-330488771 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-1565505675 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-1565505675, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:18:21.167972 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-c2ce1569-df1e-4db3-b434-56e9377aa881 tempest-GroupSnapshotsTest-330488771 None] Created reservations ['35ab5bab-0af6-4792-80d5-baa69c24c358', 'e562aa76-8699-4a09-b145-a2b87ea7dd63', 'c0963b2d-abdf-421e-80de-d3b49b5f1f53', '001186dc-0ba4-40f2-baa7-27a8f070a3b3'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:21.297201 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2ce1569-df1e-4db3-b434-56e9377aa881 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:21.347238 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2ce1569-df1e-4db3-b434-56e9377aa881 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:21.376568 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-c2ce1569-df1e-4db3-b434-56e9377aa881 tempest-GroupSnapshotsTest-330488771 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:18:21.384484 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c2ce1569-df1e-4db3-b434-56e9377aa881 tempest-GroupSnapshotsTest-330488771 None] Delete group completed successfully. Sep 17 21:18:21.934934 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2708bb40-c860-4d2b-b1f5-2310c74ae7e0 tempest-GroupSnapshotsTest-330488771 None] group_snapshot 47573027-31a0-4b3f-886b-bf0bdfcd8844: deleting Sep 17 21:18:21.954209 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2708bb40-c860-4d2b-b1f5-2310c74ae7e0 tempest-GroupSnapshotsTest-330488771 None] group_snapshot 47573027-31a0-4b3f-886b-bf0bdfcd8844: deleting {{(pid=133409) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Sep 17 21:18:21.972031 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2708bb40-c860-4d2b-b1f5-2310c74ae7e0 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:22.386394 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-2708bb40-c860-4d2b-b1f5-2310c74ae7e0 tempest-GroupSnapshotsTest-330488771 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-1565505675 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-1565505675, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:18:22.403319 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-2708bb40-c860-4d2b-b1f5-2310c74ae7e0 tempest-GroupSnapshotsTest-330488771 None] Created reservations ['3657e8de-a704-4730-a526-8e5963e9e0b8', 'ed06edf9-bba4-45eb-b5e3-dd4ddc8f59d5', 'd1c270e3-026a-4967-80ac-b5516eda6943', 'd02ded8b-c06d-4ebc-a500-4e61fa29e406'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:22.462080 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2708bb40-c860-4d2b-b1f5-2310c74ae7e0 tempest-GroupSnapshotsTest-330488771 None] group_snapshot 47573027-31a0-4b3f-886b-bf0bdfcd8844: deleted successfully Sep 17 21:18:23.490505 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b5fb74e0-f1c4-4a45-ae81-c1b5a59bc64d tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:23.558428 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b5fb74e0-f1c4-4a45-ae81-c1b5a59bc64d tempest-GroupSnapshotsTest-330488771 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:18:23.558841 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b5fb74e0-f1c4-4a45-ae81-c1b5a59bc64d tempest-GroupSnapshotsTest-330488771 None] Volume volume-ceb0b14c-98ee-4885-aa9c-e63a03575bab is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:18:23.566135 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b5fb74e0-f1c4-4a45-ae81-c1b5a59bc64d tempest-GroupSnapshotsTest-330488771 None] deleting rbd volume volume-ceb0b14c-98ee-4885-aa9c-e63a03575bab {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:18:23.812436 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=64,cinder:UPDATE=28,cinder:INSERT=8 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:23.850435 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-b5fb74e0-f1c4-4a45-ae81-c1b5a59bc64d tempest-GroupSnapshotsTest-330488771 None] Created reservations ['00d12d59-d8d7-48e4-b8fc-62e396e574a9'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:23.887445 ncdv-master-1 cinder-volume[133409]: WARNING cinder.db.sqlalchemy.api [None req-b5fb74e0-f1c4-4a45-ae81-c1b5a59bc64d tempest-GroupSnapshotsTest-330488771 None] Reservation would make usage less than 0 for the following resources, so on commit they will be limited to prevent going below 0: ['gigabytes', 'gigabytes_tempest-GroupSnapshotsTest-volume-type-1565505675'] Sep 17 21:18:23.897252 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-b5fb74e0-f1c4-4a45-ae81-c1b5a59bc64d tempest-GroupSnapshotsTest-330488771 None] Created reservations ['1d2f6e58-7da5-480f-ac26-a72df63d696c', 'e0ae2655-faaa-47f0-8d9e-001b4154ac70', '50255398-33d7-4925-a510-b864b8d3f346', '45bc3308-04ee-44c5-be01-7e2e246a43cd'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:24.017038 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b5fb74e0-f1c4-4a45-ae81-c1b5a59bc64d tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:24.093785 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b5fb74e0-f1c4-4a45-ae81-c1b5a59bc64d tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:24.158806 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-b5fb74e0-f1c4-4a45-ae81-c1b5a59bc64d tempest-GroupSnapshotsTest-330488771 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:18:24.173581 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b5fb74e0-f1c4-4a45-ae81-c1b5a59bc64d tempest-GroupSnapshotsTest-330488771 None] Delete group completed successfully. Sep 17 21:18:24.970777 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ec3d6683-f917-45c9-bdc5-e169372e6a24 tempest-GroupSnapshotsTest-330488771 None] Group tempest-GroupSnapshotsTest-Group-1620095721: creating Sep 17 21:18:24.993999 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ec3d6683-f917-45c9-bdc5-e169372e6a24 tempest-GroupSnapshotsTest-330488771 None] Group tempest-GroupSnapshotsTest-Group-1620095721: created successfully Sep 17 21:18:25.004908 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ec3d6683-f917-45c9-bdc5-e169372e6a24 tempest-GroupSnapshotsTest-330488771 None] Create group completed successfully. Sep 17 21:18:26.490629 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:18:26.523563 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Flow 'volume_create_manager' (9ad3cd30-9c0e-4184-9089-f2a3c2361d31) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:26.530876 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a39ad6a0-d1ce-48c5-bf56-e326619b3b4e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:26.548378 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a39ad6a0-d1ce-48c5-bf56-e326619b3b4e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:18:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1010658631',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=7090788b-469b-4f13-a66d-3836896b61f4,host='ncdv-master-1@ceph#ceph',id=750178ba-7b84-4377-bafd-cbcee1f726db,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f5457492c1db4888802770dcf7055c1d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:18:26Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:18:26Z,use_quota=True,user_id='df2f60fa0bde4397aaabf6caa8e4efe2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(3b579720-29ee-4eab-b1c9-42469a71bc80),volume_type_id=3b579720-29ee-4eab-b1c9-42469a71bc80)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:26.555054 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5eba33b1-3a0f-4bc7-b030-9b3e3f287980) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:26.559322 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5eba33b1-3a0f-4bc7-b030-9b3e3f287980) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:26.564605 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (530e4cd2-f802-4c5a-8747-ac700ecbd108) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:26.569499 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (530e4cd2-f802-4c5a-8747-ac700ecbd108) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '750178ba-7b84-4377-bafd-cbcee1f726db', 'volume_name': 'volume-750178ba-7b84-4377-bafd-cbcee1f726db', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:26.574628 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (80cd811f-9638-4725-8948-6cc01260275a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:26.578721 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (80cd811f-9638-4725-8948-6cc01260275a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:26.583560 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (efb8f892-d385-41b6-8ade-da636c14aad4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:26.584941 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Volume 750178ba-7b84-4377-bafd-cbcee1f726db: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-750178ba-7b84-4377-bafd-cbcee1f726db', 'volume_size': 1} Sep 17 21:18:26.585674 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] creating volume 'volume-750178ba-7b84-4377-bafd-cbcee1f726db' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:18:26.586586 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:26.675839 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:26.676425 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:26.680298 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (efb8f892-d385-41b6-8ade-da636c14aad4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-750178ba-7b84-4377-bafd-cbcee1f726db', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:26.686624 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (eb79f015-a6ec-450d-bd30-1d0475caec30) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:26.696684 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Volume volume-750178ba-7b84-4377-bafd-cbcee1f726db (750178ba-7b84-4377-bafd-cbcee1f726db): created successfully Sep 17 21:18:26.701977 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (eb79f015-a6ec-450d-bd30-1d0475caec30) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:26.706916 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Flow 'volume_create_manager' (9ad3cd30-9c0e-4184-9089-f2a3c2361d31) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:26.718320 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-991166be-b2eb-4d34-a30e-2531b7fabfb0 tempest-GroupSnapshotsTest-330488771 None] Created volume successfully. Sep 17 21:18:27.989957 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:18:28.011849 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Flow 'volume_create_manager' (0b003412-1789-4602-8ae3-0ea37e60248b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:28.018867 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7a9b9f10-cefb-4729-89b0-7935598a6bf0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:28.036097 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7a9b9f10-cefb-4729-89b0-7935598a6bf0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:18:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-855756415',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=7090788b-469b-4f13-a66d-3836896b61f4,host='ncdv-master-1@ceph#ceph',id=0a80a3a3-1ed9-46f7-83eb-f60601549a91,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f5457492c1db4888802770dcf7055c1d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:18:28Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:18:28Z,use_quota=True,user_id='df2f60fa0bde4397aaabf6caa8e4efe2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(3b579720-29ee-4eab-b1c9-42469a71bc80),volume_type_id=3b579720-29ee-4eab-b1c9-42469a71bc80)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:28.041452 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fe3eee2e-ec64-447f-9a58-1159cc367f23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:28.045496 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fe3eee2e-ec64-447f-9a58-1159cc367f23) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:28.050394 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7b932343-e796-4464-b473-3230c886f10b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:28.054416 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7b932343-e796-4464-b473-3230c886f10b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '0a80a3a3-1ed9-46f7-83eb-f60601549a91', 'volume_name': 'volume-0a80a3a3-1ed9-46f7-83eb-f60601549a91', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:28.059102 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d1d2dac3-0d0a-4868-a24d-97e8b903c968) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:28.062986 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d1d2dac3-0d0a-4868-a24d-97e8b903c968) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:28.068120 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1458ab1a-7eaa-4148-ade1-5ccf8988f71a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:28.069167 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Volume 0a80a3a3-1ed9-46f7-83eb-f60601549a91: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-0a80a3a3-1ed9-46f7-83eb-f60601549a91', 'volume_size': 1} Sep 17 21:18:28.069613 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] creating volume 'volume-0a80a3a3-1ed9-46f7-83eb-f60601549a91' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:18:28.070285 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:28.140993 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:28.141586 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:28.145523 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1458ab1a-7eaa-4148-ade1-5ccf8988f71a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-0a80a3a3-1ed9-46f7-83eb-f60601549a91', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:28.152052 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a6804444-5dfa-462c-96bf-8867b4f94c51) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:28.164395 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Volume volume-0a80a3a3-1ed9-46f7-83eb-f60601549a91 (0a80a3a3-1ed9-46f7-83eb-f60601549a91): created successfully Sep 17 21:18:28.168530 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a6804444-5dfa-462c-96bf-8867b4f94c51) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:28.177658 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Flow 'volume_create_manager' (0b003412-1789-4602-8ae3-0ea37e60248b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:28.187639 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d8df3fd8-bc5e-42c3-adb5-6b775d64527a tempest-GroupSnapshotsTest-330488771 None] Created volume successfully. Sep 17 21:18:29.211035 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=32,cinder:SELECT=62,cinder:INSERT=5,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:29.214226 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:18:29.234591 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Flow 'volume_create_manager' (5961acc3-895d-4dbd-b1c9-fb7da07fcb39) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:29.240916 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (532df6bd-7130-4872-9599-4d9b79caf48d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:29.262991 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (532df6bd-7130-4872-9599-4d9b79caf48d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:18:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-2125663784',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f2fb36b6-ba25-42a8-bf5a-89fe11d3887f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e23900d23028416480c628993067f208',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:18:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:18:29Z,use_quota=True,user_id='cc1fd61c78d04d65b4cf36052ae85970',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:29.268540 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ab3bef61-9467-4d00-bed8-16a672a3eb01) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:29.272653 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ab3bef61-9467-4d00-bed8-16a672a3eb01) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:29.277650 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1703cb56-6a66-4926-a4b3-933744033403) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:29.281547 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1703cb56-6a66-4926-a4b3-933744033403) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f2fb36b6-ba25-42a8-bf5a-89fe11d3887f', 'volume_name': 'volume-f2fb36b6-ba25-42a8-bf5a-89fe11d3887f', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:29.286213 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a30d038c-ce1b-4852-ad35-36e6103c58b2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:29.290021 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a30d038c-ce1b-4852-ad35-36e6103c58b2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:29.294499 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (600fde6c-d34c-4839-bcbd-95dd00d507d2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:29.295545 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Volume f2fb36b6-ba25-42a8-bf5a-89fe11d3887f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f2fb36b6-ba25-42a8-bf5a-89fe11d3887f', 'volume_size': 1} Sep 17 21:18:29.295930 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] creating volume 'volume-f2fb36b6-ba25-42a8-bf5a-89fe11d3887f' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:18:29.296568 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:29.345962 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-fe92257f-67a7-495f-aa97-f269667fe679 tempest-GroupSnapshotsTest-330488771 None] Update group completed successfully. Sep 17 21:18:29.383989 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:29.384825 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:29.389898 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (600fde6c-d34c-4839-bcbd-95dd00d507d2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f2fb36b6-ba25-42a8-bf5a-89fe11d3887f', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:29.396487 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (56c777f4-8236-42c7-9bcc-5159bbddf72b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:29.406803 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Volume volume-f2fb36b6-ba25-42a8-bf5a-89fe11d3887f (f2fb36b6-ba25-42a8-bf5a-89fe11d3887f): created successfully Sep 17 21:18:29.410583 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (56c777f4-8236-42c7-9bcc-5159bbddf72b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:29.416178 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Flow 'volume_create_manager' (5961acc3-895d-4dbd-b1c9-fb7da07fcb39) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:29.426544 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-192bd9c1-83da-4dd1-adbd-e2099eb1ff2d tempest-VolumesMetadataTest-1379981763 None] Created volume successfully. Sep 17 21:18:30.466115 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-75ce0910-2fd2-4c4d-bd92-1c8e2bcbdee7 tempest-GroupSnapshotsTest-330488771 None] Update group completed successfully. Sep 17 21:18:31.223275 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] Acquiring lock "cinder-f2fb36b6-ba25-42a8-bf5a-89fe11d3887f-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:18:31.249926 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] Lock "cinder-f2fb36b6-ba25-42a8-bf5a-89fe11d3887f-delete_volume" acquired by "delete_volume" :: waited 0.026s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:18:31.307627 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:31.368675 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:18:31.369937 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] Volume volume-f2fb36b6-ba25-42a8-bf5a-89fe11d3887f is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:18:31.375289 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] deleting rbd volume volume-f2fb36b6-ba25-42a8-bf5a-89fe11d3887f {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:18:31.718785 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-0ef3ea0e-0ef8-46b9-9922-98b77576ee29 tempest-GroupSnapshotsTest-330488771 None] Update group completed successfully. Sep 17 21:18:31.756195 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] Created reservations ['f8e812ba-57d5-454e-a051-b27357c468e0', '2901ac08-0cfa-4c93-a967-9f105ef96621', '0974c333-6112-4fe5-af75-390fd7f763e3', '111e3367-d73d-47fb-a4e4-6543c68dcf50'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:31.811740 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:31.842013 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:31.892192 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:18:31.898709 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] Deleted volume successfully. Sep 17 21:18:31.917635 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-aea56509-c00b-4053-b2da-05bfdcaa6fa1 tempest-VolumesMetadataTest-1379981763 None] Lock "cinder-f2fb36b6-ba25-42a8-bf5a-89fe11d3887f-delete_volume" released by "delete_volume" :: held 0.668s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:18:32.968850 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=64,cinder:INSERT=4,cinder:UPDATE=29,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:32.990509 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-feabacb8-132c-4212-8552-7b5dee482b0a tempest-GroupSnapshotsTest-330488771 None] Update group completed successfully. Sep 17 21:18:33.233104 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:18:33.300847 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Flow 'volume_create_manager' (e946f778-6af0-41cc-b2dd-606f9f4c8256) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:33.310697 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1e526ef6-293a-4443-992f-72566b0fd466) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:33.338749 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1e526ef6-293a-4443-992f-72566b0fd466) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:18:33Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-1888980762',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e7b28ecd-7573-4b77-a5cd-36f4afa22ac4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='720881c6f7304aadbf069698c97dc5c5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:18:33Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:18:33Z,use_quota=True,user_id='8ddb1d0a7bca457493d7ac9a58a01e64',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:33.343887 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1eccdbf2-0316-4f04-ada1-ffd50f9d04f8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:33.353876 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1eccdbf2-0316-4f04-ada1-ffd50f9d04f8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:33.360886 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d29f584b-dea2-4b37-9591-858ca24b2f0c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:33.365206 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d29f584b-dea2-4b37-9591-858ca24b2f0c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e7b28ecd-7573-4b77-a5cd-36f4afa22ac4', 'volume_name': 'volume-e7b28ecd-7573-4b77-a5cd-36f4afa22ac4', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:33.370729 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4f10edfd-4f9a-41a2-a04d-784f2f79dba3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:33.375855 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4f10edfd-4f9a-41a2-a04d-784f2f79dba3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:33.381849 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (77014edf-61d9-4755-b7db-052950f0a8cb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:33.382970 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Volume e7b28ecd-7573-4b77-a5cd-36f4afa22ac4: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e7b28ecd-7573-4b77-a5cd-36f4afa22ac4', 'volume_size': 1} Sep 17 21:18:33.383456 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] creating volume 'volume-e7b28ecd-7573-4b77-a5cd-36f4afa22ac4' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:18:33.384908 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:33.471171 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:33.471535 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:33.475134 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (77014edf-61d9-4755-b7db-052950f0a8cb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e7b28ecd-7573-4b77-a5cd-36f4afa22ac4', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:33.479536 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8b4ef18b-5c9a-4a32-a7e2-2b04b82011b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:33.489570 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Volume volume-e7b28ecd-7573-4b77-a5cd-36f4afa22ac4 (e7b28ecd-7573-4b77-a5cd-36f4afa22ac4): created successfully Sep 17 21:18:33.497170 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8b4ef18b-5c9a-4a32-a7e2-2b04b82011b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:33.502821 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Flow 'volume_create_manager' (e946f778-6af0-41cc-b2dd-606f9f4c8256) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:33.515960 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-575cee8f-13b1-471d-a86a-2dea74fbf26a tempest-AttachVolumeShelveTestJSON-1971975439 None] Created volume successfully. Sep 17 21:18:34.572830 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2fad2e53-d333-4a16-a8cf-cb71c26a65bc tempest-GroupSnapshotsTest-330488771 None] GroupSnapshot b63179b8-047d-4678-904a-2fe42f9204d5: creating. Sep 17 21:18:34.635670 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-2fad2e53-d333-4a16-a8cf-cb71c26a65bc tempest-GroupSnapshotsTest-330488771 None] Group snapshot b63179b8-047d-4678-904a-2fe42f9204d5: creating. {{(pid=133409) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Sep 17 21:18:34.667756 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2fad2e53-d333-4a16-a8cf-cb71c26a65bc tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:35.578882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2fad2e53-d333-4a16-a8cf-cb71c26a65bc tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:36.631557 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:18:36.655292 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (90dfb830-cf7f-4e47-8a8d-f07b0be57170) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:36.662508 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (23b8d8e6-dd04-4b3c-9054-39cbab7f009d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:36.709125 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (23b8d8e6-dd04-4b3c-9054-39cbab7f009d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:18:36Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-530263124',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5b712483-f63f-4381-9968-4e3fa55f72b6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='335955c0be3743eda2541b7d04ed2e06',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:18:37Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:18:37Z,use_quota=True,user_id='fd15b9d354eb44b0bfa83bcdcf5c1384',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:36.716220 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (52d7dd68-945f-4165-a532-a400eb014dd4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:36.720710 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (52d7dd68-945f-4165-a532-a400eb014dd4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:36.726856 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b22c9cc5-423e-4b85-b5bb-b4ad17fd8aa6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:36.731372 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b22c9cc5-423e-4b85-b5bb-b4ad17fd8aa6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5b712483-f63f-4381-9968-4e3fa55f72b6', 'volume_name': 'volume-5b712483-f63f-4381-9968-4e3fa55f72b6', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:36.738486 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fa4e2a6a-a778-4777-965f-110a8b11fb18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:36.745966 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fa4e2a6a-a778-4777-965f-110a8b11fb18) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:36.754075 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f0b3cb89-48ba-480f-b2b7-6a3868552e5e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:36.758902 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume 5b712483-f63f-4381-9968-4e3fa55f72b6: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5b712483-f63f-4381-9968-4e3fa55f72b6', 'volume_size': 1} Sep 17 21:18:36.759444 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] creating volume 'volume-5b712483-f63f-4381-9968-4e3fa55f72b6' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:18:36.761120 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:36.869883 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:36.870493 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:36.886467 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f0b3cb89-48ba-480f-b2b7-6a3868552e5e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5b712483-f63f-4381-9968-4e3fa55f72b6', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:36.886467 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d5308da1-eb8c-4498-902f-94e7225937c9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:36.900860 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume volume-5b712483-f63f-4381-9968-4e3fa55f72b6 (5b712483-f63f-4381-9968-4e3fa55f72b6): created successfully Sep 17 21:18:36.904990 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d5308da1-eb8c-4498-902f-94e7225937c9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:36.910939 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (90dfb830-cf7f-4e47-8a8d-f07b0be57170) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:36.938032 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a1865228-0738-4a28-8735-a4f2b35d6ed1 tempest-VolumesSnapshotTestJSON-2104722578 None] Created volume successfully. Sep 17 21:18:36.977780 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2fad2e53-d333-4a16-a8cf-cb71c26a65bc tempest-GroupSnapshotsTest-330488771 None] group_snapshot b63179b8-047d-4678-904a-2fe42f9204d5: created successfully Sep 17 21:18:37.880376 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1c39d710-3d6d-4d94-8d0e-c47922518ce4 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:38.065890 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-0a7c8100-4312-491f-8e11-3df669d14f52 tempest-GroupSnapshotsTest-330488771 None] group_snapshot b63179b8-047d-4678-904a-2fe42f9204d5: deleting Sep 17 21:18:38.088598 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0a7c8100-4312-491f-8e11-3df669d14f52 tempest-GroupSnapshotsTest-330488771 None] group_snapshot b63179b8-047d-4678-904a-2fe42f9204d5: deleting {{(pid=133409) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Sep 17 21:18:38.109025 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0a7c8100-4312-491f-8e11-3df669d14f52 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:38.597343 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0a7c8100-4312-491f-8e11-3df669d14f52 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:38.681714 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-1c39d710-3d6d-4d94-8d0e-c47922518ce4 tempest-VolumesSnapshotTestJSON-2104722578 None] Create snapshot completed successfully Sep 17 21:18:39.132574 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.driver [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] Creating a new backup for volume volume-5b712483-f63f-4381-9968-4e3fa55f72b6. {{(pid=133409) _get_backup_volume_temp_volume /opt/stack/cinder/cinder/volume/driver.py:1218}} Sep 17 21:18:39.164861 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] cloning volumes/volume-5b712483-f63f-4381-9968-4e3fa55f72b6@snapshot-f6a42616-014d-4a50-80ab-633d0e60c8cb to volume-88949105-c287-4383-8c20-ecffc25b5813 {{(pid=133409) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 17 21:18:39.165528 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:39.200101 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:39.306733 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:39.307161 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:39.329115 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-17T21:18:38Z,data_timestamp=2022-09-17T21:18:38Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Backup-1676633007',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=e34c31da-5f5c-46cd-803f-fc8b124f6b7c,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='335955c0be3743eda2541b7d04ed2e06',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id='f6a42616-014d-4a50-80ab-633d0e60c8cb',status='creating',temp_snapshot_id=None,temp_volume_id='88949105-c287-4383-8c20-ecffc25b5813',updated_at=None,user_id='fd15b9d354eb44b0bfa83bcdcf5c1384',volume_id=5b712483-f63f-4381-9968-4e3fa55f72b6) Sep 17 21:18:39.329944 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.backup.rpcapi [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] continue_backup in rpcapi backup_id e34c31da-5f5c-46cd-803f-fc8b124f6b7c {{(pid=133409) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 17 21:18:39.451118 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:18:39.676620 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-0a7c8100-4312-491f-8e11-3df669d14f52 tempest-GroupSnapshotsTest-330488771 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-753207063 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-753207063, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:18:39.677582 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-0a7c8100-4312-491f-8e11-3df669d14f52 tempest-GroupSnapshotsTest-330488771 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-753207063 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-753207063, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:18:39.711509 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-0a7c8100-4312-491f-8e11-3df669d14f52 tempest-GroupSnapshotsTest-330488771 None] Created reservations ['0e097d56-c538-4761-bd2a-325c898fd7e6', '0ffb4c75-5010-478c-ba20-4f67c901f5d4', 'ac0ff127-be1a-474f-883f-76e0b8320f95', 'f3531472-49a9-4bc8-b264-c95b7c732982'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:39.750096 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=71,cinder:UPDATE=24,cinder:INSERT=4,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:39.812505 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-0a7c8100-4312-491f-8e11-3df669d14f52 tempest-GroupSnapshotsTest-330488771 None] Created reservations ['689397d5-d1c5-47c7-9f03-a51b57f1b55d', 'a9a5d2a7-8c3e-4b66-b23b-c690905f3701', '34348cb7-01ef-4d21-a95a-97815b0b7fed', 'f47692ba-2d48-4d54-8123-8579bdde516a'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:39.861703 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-0a7c8100-4312-491f-8e11-3df669d14f52 tempest-GroupSnapshotsTest-330488771 None] group_snapshot b63179b8-047d-4678-904a-2fe42f9204d5: deleted successfully Sep 17 21:18:40.287686 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.837s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:18:40.288601 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-88949105-c287-4383-8c20-ecffc25b5813', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '88949105-c287-4383-8c20-ecffc25b5813', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:18:40.302907 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] Initialize volume connection completed successfully. Sep 17 21:18:40.762715 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:40.885225 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:18:40.886118 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] Volume volume-0a80a3a3-1ed9-46f7-83eb-f60601549a91 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:18:40.898800 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] deleting rbd volume volume-0a80a3a3-1ed9-46f7-83eb-f60601549a91 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:18:41.514716 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:41.598760 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:18:41.602761 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] Volume volume-750178ba-7b84-4377-bafd-cbcee1f726db is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:18:41.635153 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] deleting rbd volume volume-750178ba-7b84-4377-bafd-cbcee1f726db {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:18:41.900937 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=70,cinder:UPDATE=23,cinder:DELETE=2,cinder:INSERT=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:41.915837 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] Created reservations ['3082bbb3-53da-4877-8795-09d11b38a912'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:41.936812 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-753207063 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-753207063, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:18:41.951631 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] Created reservations ['8ecdfad0-f6ee-4622-bdd9-1380656536fb', '4984e314-74e8-46fc-9a44-5cd795f5f761', '9c38f4e9-b042-4f7d-ae06-396085797719', '83d476ba-35ca-45e3-9b9c-c7d5c3c6abfd'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:42.052746 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] Created reservations ['d26e7a59-0f0c-439b-b2e9-69ae09996a3f', '6398f76b-a2b6-4a56-ab4c-b9fa2041b12f', '3fc4ae9f-56b2-4b01-869f-8a10c79a2fc4', '61940028-969d-4624-be57-bcfd7ad6607c'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:42.198437 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:42.259375 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:42.308324 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:18:42.324881 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-4a646607-a8aa-410d-b2c9-cf594aa33346 tempest-GroupSnapshotsTest-330488771 None] Delete group completed successfully. Sep 17 21:18:43.345976 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d04b76f8-60ee-4a7b-b30a-35c023eec346 tempest-GroupSnapshotsTest-330488771 None] Group tempest-GroupSnapshotsTest-Group-1320989884: creating Sep 17 21:18:43.368346 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d04b76f8-60ee-4a7b-b30a-35c023eec346 tempest-GroupSnapshotsTest-330488771 None] Group tempest-GroupSnapshotsTest-Group-1320989884: created successfully Sep 17 21:18:43.381237 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d04b76f8-60ee-4a7b-b30a-35c023eec346 tempest-GroupSnapshotsTest-330488771 None] Create group completed successfully. Sep 17 21:18:44.842408 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:18:44.890945 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Flow 'volume_create_manager' (afd0ed66-08cd-4710-ad31-f97116d5a6ec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:44.898586 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2a25a034-b412-4536-b0ac-8b51e185bb7d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:44.918614 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2a25a034-b412-4536-b0ac-8b51e185bb7d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:18:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1037930955',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=75212706-4785-4f57-abb8-0fb315e2ad5f,host='ncdv-master-1@ceph#ceph',id=af949e6f-f9b9-4635-98bd-dcb797121ab5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f5457492c1db4888802770dcf7055c1d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:18:45Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:18:45Z,use_quota=True,user_id='df2f60fa0bde4397aaabf6caa8e4efe2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(be5abe4c-e0eb-4231-a704-23cb1d21251b),volume_type_id=be5abe4c-e0eb-4231-a704-23cb1d21251b)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:44.924287 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1a81bdfe-8d1b-4845-a993-993d8d22e9c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:44.928901 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1a81bdfe-8d1b-4845-a993-993d8d22e9c5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:44.934172 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (42f42438-0a31-4e1b-b99f-072739955006) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:44.938601 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (42f42438-0a31-4e1b-b99f-072739955006) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'af949e6f-f9b9-4635-98bd-dcb797121ab5', 'volume_name': 'volume-af949e6f-f9b9-4635-98bd-dcb797121ab5', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:44.943922 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f554e119-f407-4e48-b7cc-1cfaf7c1923b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:44.949658 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f554e119-f407-4e48-b7cc-1cfaf7c1923b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:44.982973 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fe9f7a50-e7e8-4f19-a358-c0f044822855) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:44.997837 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Volume af949e6f-f9b9-4635-98bd-dcb797121ab5: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-af949e6f-f9b9-4635-98bd-dcb797121ab5', 'volume_size': 1} Sep 17 21:18:44.998251 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] creating volume 'volume-af949e6f-f9b9-4635-98bd-dcb797121ab5' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:18:44.999068 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:45.232823 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:45.233276 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:45.238841 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fe9f7a50-e7e8-4f19-a358-c0f044822855) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-af949e6f-f9b9-4635-98bd-dcb797121ab5', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:45.244686 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c52805a3-19ce-4280-af2e-cea394c40572) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:45.275933 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Volume volume-af949e6f-f9b9-4635-98bd-dcb797121ab5 (af949e6f-f9b9-4635-98bd-dcb797121ab5): created successfully Sep 17 21:18:45.279821 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c52805a3-19ce-4280-af2e-cea394c40572) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:45.286049 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Flow 'volume_create_manager' (afd0ed66-08cd-4710-ad31-f97116d5a6ec) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:45.300070 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a6773a65-f3bf-4975-a5f4-d06b0b9ca6a8 tempest-GroupSnapshotsTest-330488771 None] Created volume successfully. Sep 17 21:18:46.176011 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-04dfaf68-5c92-42b0-be7e-df9b0e8014bc tempest-GroupSnapshotsTest-330488771 None] GroupSnapshot 86177394-d8cd-43b3-af40-3ae975c61bc6: creating. Sep 17 21:18:46.197363 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-04dfaf68-5c92-42b0-be7e-df9b0e8014bc tempest-GroupSnapshotsTest-330488771 None] Group snapshot 86177394-d8cd-43b3-af40-3ae975c61bc6: creating. {{(pid=133409) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Sep 17 21:18:46.228518 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-04dfaf68-5c92-42b0-be7e-df9b0e8014bc tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:47.108615 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=61,cinder:INSERT=7,cinder:UPDATE=31,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:47.127108 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-04dfaf68-5c92-42b0-be7e-df9b0e8014bc tempest-GroupSnapshotsTest-330488771 None] group_snapshot 86177394-d8cd-43b3-af40-3ae975c61bc6: created successfully Sep 17 21:18:47.461209 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e3253b59-dac5-430d-8958-b50f876cdc50 tempest-GroupSnapshotsTest-330488771 None] group_snapshot 86177394-d8cd-43b3-af40-3ae975c61bc6: deleting Sep 17 21:18:47.493165 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e3253b59-dac5-430d-8958-b50f876cdc50 tempest-GroupSnapshotsTest-330488771 None] group_snapshot 86177394-d8cd-43b3-af40-3ae975c61bc6: deleting {{(pid=133409) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Sep 17 21:18:47.517282 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e3253b59-dac5-430d-8958-b50f876cdc50 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:48.195376 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-e3253b59-dac5-430d-8958-b50f876cdc50 tempest-GroupSnapshotsTest-330488771 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-1164229135 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-1164229135, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:18:48.198743 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-e3253b59-dac5-430d-8958-b50f876cdc50 tempest-GroupSnapshotsTest-330488771 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-1164229135 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-1164229135, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:18:48.246706 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-e3253b59-dac5-430d-8958-b50f876cdc50 tempest-GroupSnapshotsTest-330488771 None] Created reservations ['d5b10fc4-204d-4ea6-a951-13b854b15a74', '72de6aed-44a6-452d-8d94-2dee27e8127c', 'f35c3b29-fb9a-40bb-b2b7-3aebe08bbe63', 'd0c23c63-93e4-4ccc-ab3e-aec9e18e4609'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:48.315206 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e3253b59-dac5-430d-8958-b50f876cdc50 tempest-GroupSnapshotsTest-330488771 None] group_snapshot 86177394-d8cd-43b3-af40-3ae975c61bc6: deleted successfully Sep 17 21:18:49.292973 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a763779c-aa42-4554-ba47-8c95f39aa154 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:49.405174 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a763779c-aa42-4554-ba47-8c95f39aa154 tempest-GroupSnapshotsTest-330488771 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:18:49.406017 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a763779c-aa42-4554-ba47-8c95f39aa154 tempest-GroupSnapshotsTest-330488771 None] Volume volume-af949e6f-f9b9-4635-98bd-dcb797121ab5 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:18:49.418811 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a763779c-aa42-4554-ba47-8c95f39aa154 tempest-GroupSnapshotsTest-330488771 None] deleting rbd volume volume-af949e6f-f9b9-4635-98bd-dcb797121ab5 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:18:49.631102 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] Terminate volume connection completed successfully. Sep 17 21:18:49.661946 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] Remove volume export completed successfully. Sep 17 21:18:49.733168 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-88949105-c287-4383-8c20-ecffc25b5813-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:18:49.754555 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-88949105-c287-4383-8c20-ecffc25b5813-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:18:49.775499 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:49.841354 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:18:49.851666 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] deleting rbd volume volume-88949105-c287-4383-8c20-ecffc25b5813 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:18:49.857747 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=66,cinder:INSERT=7,cinder:UPDATE=27 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:18:49.873952 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-a763779c-aa42-4554-ba47-8c95f39aa154 tempest-GroupSnapshotsTest-330488771 None] Created reservations ['6def4297-d78c-4350-a660-a7f9f1ff52d8'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:49.903375 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-a763779c-aa42-4554-ba47-8c95f39aa154 tempest-GroupSnapshotsTest-330488771 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-1164229135 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-1164229135, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:18:49.923097 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-a763779c-aa42-4554-ba47-8c95f39aa154 tempest-GroupSnapshotsTest-330488771 None] Created reservations ['88ca55dd-5bef-4649-a9e0-5756a2e21e99', 'b94f7698-94b1-41b2-8b54-162d393ad9ea', 'aa0b998a-3f81-4ad6-beeb-9fbab87a8723', '3316ffee-bb6e-42d1-85ba-bcef44e63be5'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:18:50.076542 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a763779c-aa42-4554-ba47-8c95f39aa154 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:50.153243 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a763779c-aa42-4554-ba47-8c95f39aa154 tempest-GroupSnapshotsTest-330488771 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:50.204458 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-a763779c-aa42-4554-ba47-8c95f39aa154 tempest-GroupSnapshotsTest-330488771 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:18:50.237242 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a763779c-aa42-4554-ba47-8c95f39aa154 tempest-GroupSnapshotsTest-330488771 None] Delete group completed successfully. Sep 17 21:18:50.258472 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] Deleted volume successfully. Sep 17 21:18:50.264530 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8be919a2-e781-47ad-bf5d-a126d37231ae tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-88949105-c287-4383-8c20-ecffc25b5813-delete_volume" released by "delete_volume" :: held 0.510s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:18:59.637067 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:18:59.670351 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Flow 'volume_create_manager' (49d2ac22-451a-41c3-8d5b-2b2e7a7004ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:59.679932 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (97d6b3a6-6ac5-4831-82e4-b3ac78332ffb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:59.708865 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (97d6b3a6-6ac5-4831-82e4-b3ac78332ffb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:18:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-664231792',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d9859c2a-bfcd-40d1-96d6-cd78ba924621,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='05516767841641de8a3c48312f31fa3f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:00Z,use_quota=True,user_id='18fa7550fc73419dbc0a56127289462d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:59.717259 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2e88a998-3817-4b0a-abaa-de0716f9f332) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:59.724095 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2e88a998-3817-4b0a-abaa-de0716f9f332) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:59.730727 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff8d576d-55e2-4ace-a524-58be4fb0aade) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:59.736938 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff8d576d-55e2-4ace-a524-58be4fb0aade) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd9859c2a-bfcd-40d1-96d6-cd78ba924621', 'volume_name': 'volume-d9859c2a-bfcd-40d1-96d6-cd78ba924621', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:59.744565 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (711987cf-5875-4441-a100-41abd25837c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:59.751262 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (711987cf-5875-4441-a100-41abd25837c3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:59.766695 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a2ae5492-3289-4287-acc3-3184bfb9470a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:59.767919 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Volume d9859c2a-bfcd-40d1-96d6-cd78ba924621: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d9859c2a-bfcd-40d1-96d6-cd78ba924621', 'volume_size': 1} Sep 17 21:18:59.768399 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] creating volume 'volume-d9859c2a-bfcd-40d1-96d6-cd78ba924621' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:18:59.769639 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:18:59.902680 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:18:59.903889 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:18:59.908400 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a2ae5492-3289-4287-acc3-3184bfb9470a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d9859c2a-bfcd-40d1-96d6-cd78ba924621', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:59.920799 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4ed25034-030d-4d51-a979-8504cc301667) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:18:59.939245 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Volume volume-d9859c2a-bfcd-40d1-96d6-cd78ba924621 (d9859c2a-bfcd-40d1-96d6-cd78ba924621): created successfully Sep 17 21:18:59.947820 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4ed25034-030d-4d51-a979-8504cc301667) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:18:59.961198 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Flow 'volume_create_manager' (49d2ac22-451a-41c3-8d5b-2b2e7a7004ca) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:18:59.978813 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-0ed43c38-f370-4242-bd8e-05beb550675f tempest-VolumesTestJSON-354820453 None] Created volume successfully. Sep 17 21:19:01.034732 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:19:01.035604 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:01.122012 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:01.230736 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:01.263797 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:01.284792 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Flow 'volume_create_manager' (a57644c8-b8b1-48e2-9f6a-ae93f57b40fc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:01.292355 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (34ec55ef-0d2f-4d6b-9979-82573d636da5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:01.314823 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (34ec55ef-0d2f-4d6b-9979-82573d636da5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-1584271769',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=acc2cf8b-8729-4738-a90a-506bf1efae19,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='05516767841641de8a3c48312f31fa3f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:01Z,use_quota=True,user_id='18fa7550fc73419dbc0a56127289462d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:01.320214 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0134ff69-9c24-4672-8528-28b2ab6523e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:01.324375 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0134ff69-9c24-4672-8528-28b2ab6523e1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:01.329834 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (33a4313f-6962-49d2-aafd-45764c2550c2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:01.333902 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (33a4313f-6962-49d2-aafd-45764c2550c2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'acc2cf8b-8729-4738-a90a-506bf1efae19', 'volume_name': 'volume-acc2cf8b-8729-4738-a90a-506bf1efae19', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:01.338820 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a00c6eb5-22de-47fa-b3d2-81702497eb82) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:01.343406 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a00c6eb5-22de-47fa-b3d2-81702497eb82) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:01.347958 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5a7ba0cb-ce13-4dbf-bba3-6715ddf13c52) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:01.349035 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Volume acc2cf8b-8729-4738-a90a-506bf1efae19: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-acc2cf8b-8729-4738-a90a-506bf1efae19', 'volume_size': 1} Sep 17 21:19:01.349425 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] creating volume 'volume-acc2cf8b-8729-4738-a90a-506bf1efae19' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:19:01.350147 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:01.464350 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:01.464701 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:01.469329 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5a7ba0cb-ce13-4dbf-bba3-6715ddf13c52) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-acc2cf8b-8729-4738-a90a-506bf1efae19', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:01.479436 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (98f2988c-ec3e-4646-baa5-2911a054f6e2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:01.507031 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Volume volume-acc2cf8b-8729-4738-a90a-506bf1efae19 (acc2cf8b-8729-4738-a90a-506bf1efae19): created successfully Sep 17 21:19:01.512145 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (98f2988c-ec3e-4646-baa5-2911a054f6e2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:01.519968 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Flow 'volume_create_manager' (a57644c8-b8b1-48e2-9f6a-ae93f57b40fc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:01.542936 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f3fa1111-977d-429c-bc60-e35f4115d3f9 tempest-VolumesTestJSON-354820453 None] Created volume successfully. Sep 17 21:19:02.972960 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:03.004734 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Flow 'volume_create_manager' (edf6e2b1-ab2c-4d2b-9c0e-73531b1e187c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:03.012040 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4f103c05-310c-4122-9851-5af6294f14a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:03.031963 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4f103c05-310c-4122-9851-5af6294f14a3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-1223415259',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3d594660-2431-4012-8be5-67ec0dd33f0e,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='05516767841641de8a3c48312f31fa3f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:03Z,use_quota=True,user_id='18fa7550fc73419dbc0a56127289462d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:03.037752 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b8804e04-6763-404a-a456-66cfe725bdf7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:03.042304 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b8804e04-6763-404a-a456-66cfe725bdf7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:03.047461 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5025f7d4-9022-464d-827e-94dee68dfe33) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:03.052186 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5025f7d4-9022-464d-827e-94dee68dfe33) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '3d594660-2431-4012-8be5-67ec0dd33f0e', 'volume_name': 'volume-3d594660-2431-4012-8be5-67ec0dd33f0e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:03.057220 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (451ead40-e586-4e41-8888-1c82a0c37e83) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:03.063550 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (451ead40-e586-4e41-8888-1c82a0c37e83) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:03.068745 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (00863786-ae8e-4163-9b58-1ae341b26d1b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:03.070125 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Volume 3d594660-2431-4012-8be5-67ec0dd33f0e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-3d594660-2431-4012-8be5-67ec0dd33f0e', 'volume_size': 1} Sep 17 21:19:03.070632 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] creating volume 'volume-3d594660-2431-4012-8be5-67ec0dd33f0e' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:19:03.071370 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:03.228487 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:03.229102 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:03.233090 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (00863786-ae8e-4163-9b58-1ae341b26d1b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3d594660-2431-4012-8be5-67ec0dd33f0e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:03.240677 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7b673b47-2544-4f0d-af53-b37b02e856d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:03.258718 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Volume volume-3d594660-2431-4012-8be5-67ec0dd33f0e (3d594660-2431-4012-8be5-67ec0dd33f0e): created successfully Sep 17 21:19:03.264688 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7b673b47-2544-4f0d-af53-b37b02e856d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:03.271385 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Flow 'volume_create_manager' (edf6e2b1-ab2c-4d2b-9c0e-73531b1e187c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:03.285282 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-cbba477e-447d-4e24-9f93-36e257a9b94f tempest-VolumesTestJSON-354820453 None] Created volume successfully. Sep 17 21:19:05.773388 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] Acquiring lock "cinder-3d594660-2431-4012-8be5-67ec0dd33f0e-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:05.787112 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] Lock "cinder-3d594660-2431-4012-8be5-67ec0dd33f0e-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:05.828847 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:05.910488 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:05.911189 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] Volume volume-3d594660-2431-4012-8be5-67ec0dd33f0e is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:05.918903 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] deleting rbd volume volume-3d594660-2431-4012-8be5-67ec0dd33f0e {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:06.363055 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] Created reservations ['5d4928ee-da03-489c-bec1-a3d2d675e347', '4a07bf3c-5938-4612-b934-dadd9f05b96d', 'c48c430a-b664-4370-96f4-7f0557ca1f54', '2b5b5ce7-7624-438c-ba87-647e84aa873e'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:06.384786 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=62,cinder:INSERT=9,cinder:UPDATE=28,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:06.439422 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:06.519523 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:06.579507 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:06.588566 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] Deleted volume successfully. Sep 17 21:19:06.611688 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5a1e82aa-6add-4da3-9e0b-9e62b2395f3e tempest-VolumesTestJSON-354820453 None] Lock "cinder-3d594660-2431-4012-8be5-67ec0dd33f0e-delete_volume" released by "delete_volume" :: held 0.824s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:06.946127 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] Acquiring lock "cinder-acc2cf8b-8729-4738-a90a-506bf1efae19-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:06.962789 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] Lock "cinder-acc2cf8b-8729-4738-a90a-506bf1efae19-delete_volume" acquired by "delete_volume" :: waited 0.017s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:07.027348 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:07.086616 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:07.087143 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] Volume volume-acc2cf8b-8729-4738-a90a-506bf1efae19 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:07.096454 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] deleting rbd volume volume-acc2cf8b-8729-4738-a90a-506bf1efae19 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:07.401277 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:07.446789 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (c89afd93-fe0d-4511-908f-a30f2752776e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:07.459978 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bd6c25e9-1316-42ec-87bc-43b0a3f38649) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:07.484801 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bd6c25e9-1316-42ec-87bc-43b0a3f38649) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1989485862',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f21f3370-6eb4-457e-9061-0f3b2e826e13,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='335955c0be3743eda2541b7d04ed2e06',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:07Z,use_quota=True,user_id='fd15b9d354eb44b0bfa83bcdcf5c1384',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:07.493123 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a47fb5bc-b8c7-4c4a-ab27-9107e7386d0e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:07.499446 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a47fb5bc-b8c7-4c4a-ab27-9107e7386d0e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:07.504900 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (45b01a95-f207-47b4-a14b-9294b0f98cea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:07.581090 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] Created reservations ['0f828144-c533-4072-9947-b675114fac89', '03785e7d-c5e6-477d-be43-0edf6aa517bc', 'a608a3b1-850e-4fea-86c4-ae322b06917f', '5e3835d8-8d18-421b-b8ca-8fd235058460'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:07.650662 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (45b01a95-f207-47b4-a14b-9294b0f98cea) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'f21f3370-6eb4-457e-9061-0f3b2e826e13', 'volume_name': 'volume-f21f3370-6eb4-457e-9061-0f3b2e826e13', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:07.658881 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0b52a974-ad60-432a-9678-edea528e39ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:07.663410 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0b52a974-ad60-432a-9678-edea528e39ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:07.669423 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (12ce3954-935b-4ea4-85dd-274ad5098fb6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:07.670929 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume f21f3370-6eb4-457e-9061-0f3b2e826e13: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-f21f3370-6eb4-457e-9061-0f3b2e826e13', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 21:19:07.671630 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Cloning f21f3370-6eb4-457e-9061-0f3b2e826e13 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 21:19:07.672420 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:07.687990 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.014s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:07.701830 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 19, 7, 692836)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:19:07.702567 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:19:07.706894 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 21:19:07.708895 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:07.755203 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.068s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:07.756206 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 21:19:07.759464 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:07.814021 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:07.819674 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] Deleted volume successfully. Sep 17 21:19:07.828866 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 19, 7, 825520)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:19:07.829334 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:19:07.829850 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 21:19:07.833843 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=53,cinder:INSERT=4,cinder:UPDATE=40,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:07.851597 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:07.934790 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] creating snapshot='volume-f21f3370-6eb4-457e-9061-0f3b2e826e13.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:19:08.305978 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-f21f3370-6eb4-457e-9061-0f3b2e826e13.clone_snap' to 'volume-f21f3370-6eb4-457e-9061-0f3b2e826e13' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:19:08.310146 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ecf73d44-c7e3-4ea2-85d1-dc670bc95dc3 tempest-VolumesTestJSON-354820453 None] Lock "cinder-acc2cf8b-8729-4738-a90a-506bf1efae19-delete_volume" released by "delete_volume" :: held 1.347s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:08.325786 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] Acquiring lock "cinder-d9859c2a-bfcd-40d1-96d6-cd78ba924621-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:08.341072 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] Lock "cinder-d9859c2a-bfcd-40d1-96d6-cd78ba924621-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:08.360174 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:08.380240 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Flow 'volume_create_manager' (b67c2534-df06-44e0-9aa8-de1ccfc8ec98) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:08.386863 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f91800ba-580b-4cb1-9a68-aecdcf37a111) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:08.405692 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f91800ba-580b-4cb1-9a68-aecdcf37a111) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesServicesTestJSON-Volume-1379770483',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e6f7a409-cf57-44db-ab40-bce25f0b54bf,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0dae56f733674335a6e579ed7da013e2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:08Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:08Z,use_quota=True,user_id='fc139e5ee8054df988ea2be82067172b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:08.411015 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (704432be-8b57-4106-9625-563fde6c8311) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:08.415268 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (704432be-8b57-4106-9625-563fde6c8311) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:08.420167 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (aa069874-c154-44c4-9ad4-45295a0a1dd8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:08.424455 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (aa069874-c154-44c4-9ad4-45295a0a1dd8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e6f7a409-cf57-44db-ab40-bce25f0b54bf', 'volume_name': 'volume-e6f7a409-cf57-44db-ab40-bce25f0b54bf', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:08.429228 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (414f8e22-f416-485f-affa-68284bae729d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:08.433174 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (414f8e22-f416-485f-affa-68284bae729d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:08.438207 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bed4b80c-c5bb-4b53-ac04-9a06c00c321c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:08.439357 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Volume e6f7a409-cf57-44db-ab40-bce25f0b54bf: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e6f7a409-cf57-44db-ab40-bce25f0b54bf', 'volume_size': 1} Sep 17 21:19:08.439787 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] creating volume 'volume-e6f7a409-cf57-44db-ab40-bce25f0b54bf' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:19:08.440551 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:08.509212 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:08.510711 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:08.511081 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:08.530275 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:19:08.530912 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Marking volume f21f3370-6eb4-457e-9061-0f3b2e826e13 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:19:08.543316 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:08.543972 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:08.548132 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bed4b80c-c5bb-4b53-ac04-9a06c00c321c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e6f7a409-cf57-44db-ab40-bce25f0b54bf', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:08.552723 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4e268e88-05c3-4435-8d68-e9ee74bbac85) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:08.563806 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to f21f3370-6eb4-457e-9061-0f3b2e826e13. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 21:19:08.564238 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Creating volume glance metadata for volume f21f3370-6eb4-457e-9061-0f3b2e826e13 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 21:19:08.568865 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Volume volume-e6f7a409-cf57-44db-ab40-bce25f0b54bf (e6f7a409-cf57-44db-ab40-bce25f0b54bf): created successfully Sep 17 21:19:08.572461 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4e268e88-05c3-4435-8d68-e9ee74bbac85) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:08.577763 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Flow 'volume_create_manager' (b67c2534-df06-44e0-9aa8-de1ccfc8ec98) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:08.582926 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:08.633199 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:08.633741 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] Volume volume-d9859c2a-bfcd-40d1-96d6-cd78ba924621 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:08.641678 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] deleting rbd volume volume-d9859c2a-bfcd-40d1-96d6-cd78ba924621 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:08.650773 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8e880de7-1857-452b-ba82-e3c264ffff44 tempest-VolumesServicesTestJSON-1937095819 None] Created volume successfully. Sep 17 21:19:08.707017 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (12ce3954-935b-4ea4-85dd-274ad5098fb6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f21f3370-6eb4-457e-9061-0f3b2e826e13', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:08.711832 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fc4c70ab-892c-4c0d-8cc3-ebe0d46d6757) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:08.721527 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume volume-f21f3370-6eb4-457e-9061-0f3b2e826e13 (f21f3370-6eb4-457e-9061-0f3b2e826e13): created successfully Sep 17 21:19:08.726392 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fc4c70ab-892c-4c0d-8cc3-ebe0d46d6757) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:08.732424 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (c89afd93-fe0d-4511-908f-a30f2752776e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:08.743789 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7547e4aa-dbd8-4231-a4d3-b27388b72d09 tempest-VolumesSnapshotTestJSON-2104722578 None] Created volume successfully. Sep 17 21:19:08.948545 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] Created reservations ['eeae3fb4-34d0-4960-8f5e-5d91f680654c', '37a717e2-ffb0-4e48-9114-4d0abb63dc84', '5378b1eb-0e71-48d4-8fe4-21455f36f32f', '93dcd3f6-1ac0-4c03-b660-e23eb95dc83c'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:09.013949 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:09.047058 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:09.083510 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:09.096709 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] Deleted volume successfully. Sep 17 21:19:09.113043 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-134d3015-63a1-4636-9393-94d9de2e5e7e tempest-VolumesTestJSON-354820453 None] Lock "cinder-d9859c2a-bfcd-40d1-96d6-cd78ba924621-delete_volume" released by "delete_volume" :: held 0.772s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:09.419942 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] Acquiring lock "cinder-e6f7a409-cf57-44db-ab40-bce25f0b54bf-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:09.437431 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] Lock "cinder-e6f7a409-cf57-44db-ab40-bce25f0b54bf-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:09.490698 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:09.562646 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:09.562646 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] Volume volume-e6f7a409-cf57-44db-ab40-bce25f0b54bf is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:09.573791 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] deleting rbd volume volume-e6f7a409-cf57-44db-ab40-bce25f0b54bf {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:09.948169 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] Created reservations ['0ced29ec-2cfb-4746-8dff-1d46ae8a28c8', '4119708c-b3e9-492e-bb15-c03bd7c2b5b2', 'caf1744c-1c48-4265-823d-afb763e22f93', '5015ed17-8950-4140-9058-a180f46dd355'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:10.042761 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:10.119925 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:10.207057 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:10.214798 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] Deleted volume successfully. Sep 17 21:19:10.222881 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:DELETE=2,cinder:SELECT=62,cinder:INSERT=10,cinder:UPDATE=26 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:10.246131 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-cad7952b-9409-49a2-b9d6-54b4d352b3d8 tempest-VolumesServicesTestJSON-1937095819 None] Lock "cinder-e6f7a409-cf57-44db-ab40-bce25f0b54bf-delete_volume" released by "delete_volume" :: held 0.811s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:10.348882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-392d0ccd-adee-4740-956a-0f6fca920533 req-deeabdbd-a64c-4a3e-927b-81c4977f13b9 tempest-AttachVolumeShelveTestJSON-1971975439 None] No connector for attachment f3cb2936-f9da-4454-9b81-8b70493e3397; skipping storage backend terminate_connection call. {{(pid=133409) _connection_terminate /opt/stack/cinder/cinder/volume/manager.py:4953}} Sep 17 21:19:10.349635 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-392d0ccd-adee-4740-956a-0f6fca920533 req-deeabdbd-a64c-4a3e-927b-81c4977f13b9 tempest-AttachVolumeShelveTestJSON-1971975439 None] Deleting attachment f3cb2936-f9da-4454-9b81-8b70493e3397. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:19:10.543941 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-a608543a-2222-4a5b-8482-29f6a713879b req-6bfb29d3-6294-4d6b-baf3-5c4c02e9d0a3 tempest-VolumesSnapshotTestJSON-2104722578 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:19:11.614844 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-a608543a-2222-4a5b-8482-29f6a713879b req-6bfb29d3-6294-4d6b-baf3-5c4c02e9d0a3 tempest-VolumesSnapshotTestJSON-2104722578 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.070s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:19:11.615785 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-a608543a-2222-4a5b-8482-29f6a713879b req-6bfb29d3-6294-4d6b-baf3-5c4c02e9d0a3 tempest-VolumesSnapshotTestJSON-2104722578 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f21f3370-6eb4-457e-9061-0f3b2e826e13', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'f21f3370-6eb4-457e-9061-0f3b2e826e13', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:19:11.646874 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-a608543a-2222-4a5b-8482-29f6a713879b req-6bfb29d3-6294-4d6b-baf3-5c4c02e9d0a3 tempest-VolumesSnapshotTestJSON-2104722578 None] Connection info returned from driver {'name': 'volumes/volume-f21f3370-6eb4-457e-9061-0f3b2e826e13', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'f21f3370-6eb4-457e-9061-0f3b2e826e13', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '73094c43-5e1a-43c9-ab80-7e98a3223b91'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:19:11.769319 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-a608543a-2222-4a5b-8482-29f6a713879b req-6bfb29d3-6294-4d6b-baf3-5c4c02e9d0a3 tempest-VolumesSnapshotTestJSON-2104722578 None] attachment_update completed successfully. Sep 17 21:19:13.978066 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3348803e-d25b-42aa-80da-e4f22787afa5 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:14.563767 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=52,cinder:UPDATE=25,cinder:INSERT=20,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:14.595581 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3348803e-d25b-42aa-80da-e4f22787afa5 tempest-VolumesSnapshotTestJSON-2104722578 None] Create snapshot completed successfully Sep 17 21:19:15.206922 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-784c95dc-9071-43e9-b7ab-58f02ed260d7 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:16.342032 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-784c95dc-9071-43e9-b7ab-58f02ed260d7 tempest-VolumesSnapshotTestJSON-2104722578 None] Create snapshot completed successfully Sep 17 21:19:17.605644 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:17.695916 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Flow 'volume_create_manager' (76827d09-8da4-4656-8f95-c637f2e1413b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:17.713780 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2c5d23ef-0790-4c7f-8f60-426521957c10) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:17.787130 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2c5d23ef-0790-4c7f-8f60-426521957c10) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:17Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeTypesTest-volume-242084278',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e5fe8c00-0894-4c0e-b398-33aa821a484b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d4a92ab682ff4f62bf589544eee3e0f4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:17Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:17Z,use_quota=True,user_id='608aaeabfdb24796832640f803c01abc',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(7ad55f8e-2884-4f81-875b-12a69d98a5e5),volume_type_id=7ad55f8e-2884-4f81-875b-12a69d98a5e5)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:17.800394 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7b14f017-cad8-4abe-a6cc-abc095179a3c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:17.813009 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7b14f017-cad8-4abe-a6cc-abc095179a3c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:17.819019 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (15a787f4-fa88-47c8-a4bc-bd0bef1a3e9f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:17.823147 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (15a787f4-fa88-47c8-a4bc-bd0bef1a3e9f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e5fe8c00-0894-4c0e-b398-33aa821a484b', 'volume_name': 'volume-e5fe8c00-0894-4c0e-b398-33aa821a484b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:17.828480 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b8a40c78-8373-49f5-a3b2-5c631183475b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:17.832308 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b8a40c78-8373-49f5-a3b2-5c631183475b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:17.837394 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (446d2d8d-9c35-4e88-91ba-fc7df3ea63aa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:17.838906 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Volume e5fe8c00-0894-4c0e-b398-33aa821a484b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e5fe8c00-0894-4c0e-b398-33aa821a484b', 'volume_size': 1} Sep 17 21:19:17.839319 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] creating volume 'volume-e5fe8c00-0894-4c0e-b398-33aa821a484b' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:19:17.840193 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:17.948702 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ebc634ef-8fb5-41e9-937d-e6ae117e7cfa tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:18.220324 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] extra_specs: {'storage_protocol': 'ceph', 'vendor_name': 'Open Source'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:18.221029 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] extra_specs: {'storage_protocol': 'ceph', 'vendor_name': 'Open Source'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:18.225628 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (446d2d8d-9c35-4e88-91ba-fc7df3ea63aa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e5fe8c00-0894-4c0e-b398-33aa821a484b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:18.236560 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cb5a87d1-cc68-4edb-8aff-9b62ab435c46) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:18.259459 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Volume volume-e5fe8c00-0894-4c0e-b398-33aa821a484b (e5fe8c00-0894-4c0e-b398-33aa821a484b): created successfully Sep 17 21:19:18.264399 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cb5a87d1-cc68-4edb-8aff-9b62ab435c46) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:18.276442 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Flow 'volume_create_manager' (76827d09-8da4-4656-8f95-c637f2e1413b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:18.293639 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-4fdcbf30-db0e-48a6-b9e4-ec0c189facfa tempest-VolumeTypesTest-1319480140 None] Created volume successfully. Sep 17 21:19:18.716909 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ebc634ef-8fb5-41e9-937d-e6ae117e7cfa tempest-VolumesSnapshotTestJSON-2104722578 None] Create snapshot completed successfully Sep 17 21:19:19.075665 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f545957f-53cc-4224-846c-66a037e5b0a9 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-9380873c-f7d7-4333-8f51-97d1045f7951-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:19.101730 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f545957f-53cc-4224-846c-66a037e5b0a9 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-9380873c-f7d7-4333-8f51-97d1045f7951-delete_snapshot" acquired by "delete_snapshot" :: waited 0.026s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:19.104093 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f545957f-53cc-4224-846c-66a037e5b0a9 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:19.374459 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-631850ce-26ab-48ee-ad5c-9f66f7f0e22b tempest-VolumeTypesTest-1319480140 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:19.468885 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-631850ce-26ab-48ee-ad5c-9f66f7f0e22b tempest-VolumeTypesTest-1319480140 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:19.546033 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-631850ce-26ab-48ee-ad5c-9f66f7f0e22b tempest-VolumeTypesTest-1319480140 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:19.564828 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-631850ce-26ab-48ee-ad5c-9f66f7f0e22b tempest-VolumeTypesTest-1319480140 None] Retype volume completed successfully. Sep 17 21:19:21.737496 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] Acquiring lock "cinder-e5fe8c00-0894-4c0e-b398-33aa821a484b-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:21.763898 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] Lock "cinder-e5fe8c00-0894-4c0e-b398-33aa821a484b-delete_volume" acquired by "delete_volume" :: waited 0.026s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:21.837232 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:21.913830 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:21.914219 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] Volume volume-e5fe8c00-0894-4c0e-b398-33aa821a484b is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:21.923572 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] deleting rbd volume volume-e5fe8c00-0894-4c0e-b398-33aa821a484b {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:22.380505 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-747179411 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-747179411, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:19:22.381925 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-747179411 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-747179411, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:19:22.409452 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] Created reservations ['6e94bfa7-c8f8-4978-96e0-1f9943356a41', 'f291b59e-9540-435e-8bab-d33652d11ac5', '5d59b6db-e961-40e9-8409-f9c8c18954e9', '49044afa-3c44-46a3-919a-1201ff25b7b3'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:22.432400 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=65,cinder:UPDATE=15,cinder:INSERT=19,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:22.514343 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:22.620265 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:22.727652 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:22.743197 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] Deleted volume successfully. Sep 17 21:19:22.785812 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-bae1d276-0332-46b9-a793-62ffbf2961cc tempest-VolumeTypesTest-1319480140 None] Lock "cinder-e5fe8c00-0894-4c0e-b398-33aa821a484b-delete_volume" released by "delete_volume" :: held 1.022s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:30.389155 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:30.414320 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (a69a5238-240e-4da3-b655-f9a5226cf352) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:30.423149 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6b2d93c0-e0cb-4e64-86e4-ba11031614b5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:30.440381 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6b2d93c0-e0cb-4e64-86e4-ba11031614b5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-197503520',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=904346dc-d68e-4cc4-992b-19f169f59e6d,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='c8d2ef6638184f178a760075af39cc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:30Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:30Z,use_quota=True,user_id='53aa66e4f5134dcf9adb7aa238ba43de',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:30.445993 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ad356d41-ed60-4cb0-87f5-3e5a80530fb5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:30.451046 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ad356d41-ed60-4cb0-87f5-3e5a80530fb5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:30.456338 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bfbe0da5-640f-42e4-aaca-7d4c8cd58956) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:30.460741 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bfbe0da5-640f-42e4-aaca-7d4c8cd58956) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '904346dc-d68e-4cc4-992b-19f169f59e6d', 'volume_name': 'volume-904346dc-d68e-4cc4-992b-19f169f59e6d', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:30.465537 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9974dc7a-e130-40b3-915a-7c4bf77ef05b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:30.469605 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9974dc7a-e130-40b3-915a-7c4bf77ef05b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:30.476080 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4aa4582c-ed42-4e60-bf2f-981c4c51e57d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:30.477335 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Volume 904346dc-d68e-4cc4-992b-19f169f59e6d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-904346dc-d68e-4cc4-992b-19f169f59e6d', 'volume_size': 1} Sep 17 21:19:30.477913 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] creating volume 'volume-904346dc-d68e-4cc4-992b-19f169f59e6d' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:19:30.478901 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:30.583877 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:30.584291 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:30.589324 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4aa4582c-ed42-4e60-bf2f-981c4c51e57d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-904346dc-d68e-4cc4-992b-19f169f59e6d', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:30.596371 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (37d9a0dd-3d6e-4e97-aab6-84bc7fe167dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:30.607313 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Volume volume-904346dc-d68e-4cc4-992b-19f169f59e6d (904346dc-d68e-4cc4-992b-19f169f59e6d): created successfully Sep 17 21:19:30.617065 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (37d9a0dd-3d6e-4e97-aab6-84bc7fe167dd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:30.624221 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (a69a5238-240e-4da3-b655-f9a5226cf352) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:30.634301 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-45c2b013-ebf5-4022-b4a1-f02978ba9cce tempest-VolumesGetTest-890360817 None] Created volume successfully. Sep 17 21:19:31.965231 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:31.983887 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (42328316-0a40-4e4b-ba8f-e0fdfe072dfc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:31.991570 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (59dab86e-80ae-449d-846e-e7c4c328660a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:32.008008 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (59dab86e-80ae-449d-846e-e7c4c328660a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:32Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-1473737468',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=39630834-586d-45f4-ac40-6cc0be29622b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c8d2ef6638184f178a760075af39cc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:32Z,use_quota=True,user_id='53aa66e4f5134dcf9adb7aa238ba43de',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:32.013134 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ca3e8cdc-97e3-4266-bb5d-9d08014b57e0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:32.017105 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ca3e8cdc-97e3-4266-bb5d-9d08014b57e0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:32.022126 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8715409e-ddcb-4e2e-804b-68f83fa4d30e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:32.026019 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8715409e-ddcb-4e2e-804b-68f83fa4d30e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '39630834-586d-45f4-ac40-6cc0be29622b', 'volume_name': 'volume-39630834-586d-45f4-ac40-6cc0be29622b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:32.030689 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5b620cf8-06e8-45cb-9f99-59917628426e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:32.034436 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5b620cf8-06e8-45cb-9f99-59917628426e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:32.038941 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c9151287-6cfa-4689-8d50-ab6fb650a660) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:32.039972 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Volume 39630834-586d-45f4-ac40-6cc0be29622b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-39630834-586d-45f4-ac40-6cc0be29622b', 'volume_size': 1} Sep 17 21:19:32.040352 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] creating volume 'volume-39630834-586d-45f4-ac40-6cc0be29622b' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:19:32.040989 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:32.100900 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:32.101328 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:32.106165 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c9151287-6cfa-4689-8d50-ab6fb650a660) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-39630834-586d-45f4-ac40-6cc0be29622b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:32.110922 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e4db1438-ae00-42f6-a98d-ebdd35479c15) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:32.122641 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Volume volume-39630834-586d-45f4-ac40-6cc0be29622b (39630834-586d-45f4-ac40-6cc0be29622b): created successfully Sep 17 21:19:32.127088 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e4db1438-ae00-42f6-a98d-ebdd35479c15) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:32.132550 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (42328316-0a40-4e4b-ba8f-e0fdfe072dfc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:32.141921 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-964e8c41-1248-4b05-aa48-4f0f0792b360 tempest-VolumesGetTest-890360817 None] Created volume successfully. Sep 17 21:19:33.081067 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] Acquiring lock "cinder-39630834-586d-45f4-ac40-6cc0be29622b-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:33.104527 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] Lock "cinder-39630834-586d-45f4-ac40-6cc0be29622b-delete_volume" acquired by "delete_volume" :: waited 0.023s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:33.147639 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:33.191337 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:33.191772 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] Volume volume-39630834-586d-45f4-ac40-6cc0be29622b is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:33.197739 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] deleting rbd volume volume-39630834-586d-45f4-ac40-6cc0be29622b {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:33.424550 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=61,cinder:DELETE=4,cinder:UPDATE=22,cinder:INSERT=13 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:33.437510 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] Created reservations ['2988bc3c-be8e-4b64-a01b-35787469e060', '75b45152-72ae-4b77-aacd-343d1f924a2e', '79cb4ec5-ae70-4805-8845-6a8c1be223be', 'db332b43-3845-40da-8345-e368b038b2a0'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:33.490832 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:33.536507 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:33.581079 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:33.586461 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] Deleted volume successfully. Sep 17 21:19:33.603406 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ead9decf-883e-47c3-9dc4-86bb4e01cfab tempest-VolumesGetTest-890360817 None] Lock "cinder-39630834-586d-45f4-ac40-6cc0be29622b-delete_volume" released by "delete_volume" :: held 0.499s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:34.247807 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] Acquiring lock "cinder-904346dc-d68e-4cc4-992b-19f169f59e6d-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:34.263520 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] Lock "cinder-904346dc-d68e-4cc4-992b-19f169f59e6d-delete_volume" acquired by "delete_volume" :: waited 0.017s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:34.301863 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:34.347023 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:34.347495 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] Volume volume-904346dc-d68e-4cc4-992b-19f169f59e6d is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:34.355074 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] deleting rbd volume volume-904346dc-d68e-4cc4-992b-19f169f59e6d {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:34.614127 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] Created reservations ['69280fe7-bb36-41bb-9b20-3a55577dccb8', 'a6700fb1-43f8-4975-ac54-858e70b23cb3', '2a325a1a-4277-471c-98a9-0a77a864bca3', '91986bd1-db25-483b-a462-9b8a274e6064'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:34.659555 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:34.701524 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:34.740763 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:34.746965 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] Deleted volume successfully. Sep 17 21:19:34.764478 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2031a231-1e7a-4755-a03e-ebbc84a644eb tempest-VolumesGetTest-890360817 None] Lock "cinder-904346dc-d68e-4cc4-992b-19f169f59e6d-delete_volume" released by "delete_volume" :: held 0.501s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:35.682217 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:35.701060 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (93c5b5f7-7978-400d-9074-724bd3b976e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:35.707243 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e9603b12-742d-4895-8ca2-0cdcf6aaed4b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:35.723057 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e9603b12-742d-4895-8ca2-0cdcf6aaed4b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:35Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-180762967',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ac863d25-4c41-4b94-80e8-d03560d4e42e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c8d2ef6638184f178a760075af39cc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:36Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:36Z,use_quota=True,user_id='53aa66e4f5134dcf9adb7aa238ba43de',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:35.728115 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9773b349-8e11-4b14-bf8f-79644b80c080) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:35.732252 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9773b349-8e11-4b14-bf8f-79644b80c080) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:35.737099 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ec15ff8b-5d42-40ea-a8b5-7b7950b67969) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:35.740998 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ec15ff8b-5d42-40ea-a8b5-7b7950b67969) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ac863d25-4c41-4b94-80e8-d03560d4e42e', 'volume_name': 'volume-ac863d25-4c41-4b94-80e8-d03560d4e42e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:35.746548 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cfacb107-6e67-44a8-b10d-c29eb89bdeb2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:35.751792 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cfacb107-6e67-44a8-b10d-c29eb89bdeb2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:35.760613 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (17805a48-5582-4b11-abb3-ec171b0a0b0f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:35.761847 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Volume ac863d25-4c41-4b94-80e8-d03560d4e42e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ac863d25-4c41-4b94-80e8-d03560d4e42e', 'volume_size': 1} Sep 17 21:19:35.762126 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] creating volume 'volume-ac863d25-4c41-4b94-80e8-d03560d4e42e' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:19:35.763319 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:35.830275 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:35.830646 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:35.834407 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (17805a48-5582-4b11-abb3-ec171b0a0b0f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ac863d25-4c41-4b94-80e8-d03560d4e42e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:35.838948 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c584d2b7-c50e-4812-b31d-bde26d6d1668) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:35.849582 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Volume volume-ac863d25-4c41-4b94-80e8-d03560d4e42e (ac863d25-4c41-4b94-80e8-d03560d4e42e): created successfully Sep 17 21:19:35.853380 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c584d2b7-c50e-4812-b31d-bde26d6d1668) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:35.859222 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (93c5b5f7-7978-400d-9074-724bd3b976e1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:35.868118 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c75a0fb6-2b88-4ee7-8cf5-d02de480eabb tempest-VolumesGetTest-890360817 None] Created volume successfully. Sep 17 21:19:37.062075 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:37.091439 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (8e9e2e31-c6eb-4432-ad51-96b1d17212df) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:37.097780 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (da79b288-dc17-4769-a75b-e646322a62f5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:37.114948 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (da79b288-dc17-4769-a75b-e646322a62f5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:37Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1501117841',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=aa696386-1475-49c6-a4ef-c3db0ae20bc8,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='c8d2ef6638184f178a760075af39cc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:37Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=ac863d25-4c41-4b94-80e8-d03560d4e42e,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:37Z,use_quota=True,user_id='53aa66e4f5134dcf9adb7aa238ba43de',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:37.120526 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b0e8d03b-2e39-45a0-b8b4-7d5cdfe46e84) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:37.124584 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b0e8d03b-2e39-45a0-b8b4-7d5cdfe46e84) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:37.129643 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8b9281c7-33af-48f9-9ef8-7ddee71a8ab5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:37.146379 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8b9281c7-33af-48f9-9ef8-7ddee71a8ab5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': 'aa696386-1475-49c6-a4ef-c3db0ae20bc8', 'volume_name': 'volume-aa696386-1475-49c6-a4ef-c3db0ae20bc8', 'volume_size': 1, 'source_volid': 'ac863d25-4c41-4b94-80e8-d03560d4e42e', 'source_volstatus': 'available'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:37.152347 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac97a013-aa0d-41e3-8326-16dad4ad4da3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:37.156284 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac97a013-aa0d-41e3-8326-16dad4ad4da3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:37.160920 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a25325df-1de7-4807-8e63-9b55926e747d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:37.162004 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Volume aa696386-1475-49c6-a4ef-c3db0ae20bc8: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-aa696386-1475-49c6-a4ef-c3db0ae20bc8', 'volume_size': 1, 'source_volid': 'ac863d25-4c41-4b94-80e8-d03560d4e42e', 'source_volstatus': 'available'} Sep 17 21:19:37.176900 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:37.219941 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] creating snapshot='volume-aa696386-1475-49c6-a4ef-c3db0ae20bc8.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:19:38.205887 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] cloning 'volume-ac863d25-4c41-4b94-80e8-d03560d4e42e@volume-aa696386-1475-49c6-a4ef-c3db0ae20bc8.clone_snap' to 'volume-aa696386-1475-49c6-a4ef-c3db0ae20bc8' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:19:38.302833 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Volume volume-ac863d25-4c41-4b94-80e8-d03560d4e42e is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:38.304410 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:38.304833 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:38.324429 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:19:38.328692 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a25325df-1de7-4807-8e63-9b55926e747d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-aa696386-1475-49c6-a4ef-c3db0ae20bc8', 'volume_size': 1, 'source_volid': 'ac863d25-4c41-4b94-80e8-d03560d4e42e', 'source_volstatus': 'available'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:38.333258 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7921d316-d129-432e-a004-aa8c70f058dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:38.343310 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Volume volume-aa696386-1475-49c6-a4ef-c3db0ae20bc8 (aa696386-1475-49c6-a4ef-c3db0ae20bc8): created successfully Sep 17 21:19:38.347538 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7921d316-d129-432e-a004-aa8c70f058dd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:38.353160 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (8e9e2e31-c6eb-4432-ad51-96b1d17212df) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:38.366985 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3696329a-c045-4abc-b3d5-d1a6771e2ffa tempest-VolumesGetTest-890360817 None] Created volume successfully. Sep 17 21:19:39.915762 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:39.934865 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (df46720f-9278-4ffc-a054-c0594940c494) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:39.940984 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ac71f59a-ad1a-4de9-9d07-eb2e2adc6dc6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:39.957560 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ac71f59a-ad1a-4de9-9d07-eb2e2adc6dc6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:40Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-1853440634',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c6eca159-aab2-4ee1-8227-43fb33d6c2a1,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c8d2ef6638184f178a760075af39cc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:40Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:40Z,use_quota=True,user_id='53aa66e4f5134dcf9adb7aa238ba43de',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:39.962829 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5527cda6-54c1-434c-bb8a-5dd272b7d493) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:39.966965 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5527cda6-54c1-434c-bb8a-5dd272b7d493) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:39.971871 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23ee0af9-4823-4acd-a55b-572ac0df838a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:39.975892 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23ee0af9-4823-4acd-a55b-572ac0df838a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c6eca159-aab2-4ee1-8227-43fb33d6c2a1', 'volume_name': 'volume-c6eca159-aab2-4ee1-8227-43fb33d6c2a1', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:39.980946 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5a0d1112-2540-4832-8d40-218e1fef192e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:39.984712 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5a0d1112-2540-4832-8d40-218e1fef192e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:39.989300 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bca12933-8849-412a-8745-5966cc117009) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:39.990411 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Volume c6eca159-aab2-4ee1-8227-43fb33d6c2a1: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c6eca159-aab2-4ee1-8227-43fb33d6c2a1', 'volume_size': 1} Sep 17 21:19:39.990786 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] creating volume 'volume-c6eca159-aab2-4ee1-8227-43fb33d6c2a1' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:19:39.991452 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:40.053599 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:40.054219 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:40.058426 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bca12933-8849-412a-8745-5966cc117009) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c6eca159-aab2-4ee1-8227-43fb33d6c2a1', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:40.062553 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0585ac04-dcc8-4662-8abe-d67926ec0cb2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:40.072700 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Volume volume-c6eca159-aab2-4ee1-8227-43fb33d6c2a1 (c6eca159-aab2-4ee1-8227-43fb33d6c2a1): created successfully Sep 17 21:19:40.076719 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0585ac04-dcc8-4662-8abe-d67926ec0cb2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:40.082032 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (df46720f-9278-4ffc-a054-c0594940c494) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:40.090551 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-27488d22-8af7-42e1-a47d-c35545edca76 tempest-VolumesGetTest-890360817 None] Created volume successfully. Sep 17 21:19:41.037755 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] Acquiring lock "cinder-c6eca159-aab2-4ee1-8227-43fb33d6c2a1-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:41.049876 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] Lock "cinder-c6eca159-aab2-4ee1-8227-43fb33d6c2a1-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:41.052662 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=29,cinder:DELETE=5,cinder:SELECT=61,cinder:INSERT=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:41.090622 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:41.161567 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:41.161998 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] Volume volume-c6eca159-aab2-4ee1-8227-43fb33d6c2a1 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:41.169626 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] deleting rbd volume volume-c6eca159-aab2-4ee1-8227-43fb33d6c2a1 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:41.398848 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] Created reservations ['5588102e-f3ad-4a93-b391-7621b36b5005', '24ee942d-9c3f-4b2e-91f8-39412059c3d7', '3ebde78a-feb2-483e-bff0-402f6e275da0', '2f77804e-1392-4d3e-85e7-6fbcdb967920'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:41.446690 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:41.472964 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:41.504844 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:41.510999 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] Deleted volume successfully. Sep 17 21:19:41.523674 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-c0529941-cdba-4ca2-9d49-5f4149543599 tempest-VolumesGetTest-890360817 None] Lock "cinder-c6eca159-aab2-4ee1-8227-43fb33d6c2a1-delete_volume" released by "delete_volume" :: held 0.474s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:42.223198 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] Acquiring lock "cinder-aa696386-1475-49c6-a4ef-c3db0ae20bc8-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:42.238766 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] Lock "cinder-aa696386-1475-49c6-a4ef-c3db0ae20bc8-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:42.297698 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:42.351981 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:42.362200 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] deleting rbd volume volume-aa696386-1475-49c6-a4ef-c3db0ae20bc8 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:42.603460 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:19:42.622500 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] Volume volume-ac863d25-4c41-4b94-80e8-d03560d4e42e is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:42.623015 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] deleting parent snapshot volume-aa696386-1475-49c6-a4ef-c3db0ae20bc8.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:19:43.319965 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] Created reservations ['13449918-7947-49aa-83d7-0bbe8c58a2f1', '0a17d35e-4580-41ff-997e-ca0930edd959', 'fa0f28f5-acd6-44cd-ae4d-4dbbed8e6eb1', '36f82a17-525f-4f47-a27b-c2162f777b0c'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:43.369659 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:43.394939 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:43.424875 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:43.429995 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] Deleted volume successfully. Sep 17 21:19:43.442946 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7844b092-ad45-408a-afd7-241e12880e96 tempest-VolumesGetTest-890360817 None] Lock "cinder-aa696386-1475-49c6-a4ef-c3db0ae20bc8-delete_volume" released by "delete_volume" :: held 1.204s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:44.895432 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:44.914722 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (a7c4a598-02d3-4bbc-8950-f6b0c01315e5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:44.920984 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (943809ad-836f-49f9-b1f0-d6e72729476b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:44.925580 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=34,cinder:SELECT=56,cinder:DELETE=2,cinder:INSERT=8 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:44.937529 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (943809ad-836f-49f9-b1f0-d6e72729476b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1057379244',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=feff5317-c25c-46fd-ab12-9e4fc570ae56,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='c8d2ef6638184f178a760075af39cc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:45Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:45Z,use_quota=True,user_id='53aa66e4f5134dcf9adb7aa238ba43de',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:44.942727 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0b84429b-f3a4-464f-a54a-625f33d00350) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:44.946859 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0b84429b-f3a4-464f-a54a-625f33d00350) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:44.951724 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3288cbec-4ca4-48dd-8334-ba1daa93fde0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:45.048116 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3288cbec-4ca4-48dd-8334-ba1daa93fde0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'feff5317-c25c-46fd-ab12-9e4fc570ae56', 'volume_name': 'volume-feff5317-c25c-46fd-ab12-9e4fc570ae56', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:45.054119 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (df48aa6b-c3f5-415a-8bbd-cd942ab133bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:45.057558 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (df48aa6b-c3f5-415a-8bbd-cd942ab133bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:45.062118 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (eb457de5-7ca5-4e6f-a795-e971c94dceff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:45.063234 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Volume feff5317-c25c-46fd-ab12-9e4fc570ae56: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-feff5317-c25c-46fd-ab12-9e4fc570ae56', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 21:19:45.063847 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Cloning feff5317-c25c-46fd-ab12-9e4fc570ae56 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 21:19:45.064668 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:45.080216 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:45.090720 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 19, 45, 84965)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:19:45.091194 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:19:45.091602 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 21:19:45.095917 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:45.096819 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 21:19:45.102794 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 19, 45, 100373)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:19:45.103201 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:19:45.103590 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 21:19:45.115784 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:45.172169 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] creating snapshot='volume-feff5317-c25c-46fd-ab12-9e4fc570ae56.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:19:46.133588 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-feff5317-c25c-46fd-ab12-9e4fc570ae56.clone_snap' to 'volume-feff5317-c25c-46fd-ab12-9e4fc570ae56' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:19:46.294846 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:46.296210 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:46.296650 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:46.318518 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:19:46.318970 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Marking volume feff5317-c25c-46fd-ab12-9e4fc570ae56 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:19:46.327656 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to feff5317-c25c-46fd-ab12-9e4fc570ae56. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 21:19:46.328172 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Creating volume glance metadata for volume feff5317-c25c-46fd-ab12-9e4fc570ae56 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 21:19:46.382349 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (eb457de5-7ca5-4e6f-a795-e971c94dceff) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-feff5317-c25c-46fd-ab12-9e4fc570ae56', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:46.386976 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6784f813-1b95-4833-872f-2cd7ec4a86f0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:46.396579 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Volume volume-feff5317-c25c-46fd-ab12-9e4fc570ae56 (feff5317-c25c-46fd-ab12-9e4fc570ae56): created successfully Sep 17 21:19:46.400321 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6784f813-1b95-4833-872f-2cd7ec4a86f0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:46.405755 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (a7c4a598-02d3-4bbc-8950-f6b0c01315e5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:46.414641 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b80f23b6-6f94-444f-8e91-27da2498594a tempest-VolumesGetTest-890360817 None] Created volume successfully. Sep 17 21:19:47.516729 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:19:47.536213 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (db778862-c585-455c-b967-3ccc3ee00a70) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:47.542669 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (aaf153b1-152e-48d9-b9d1-29bbb30a623c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:47.560526 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (aaf153b1-152e-48d9-b9d1-29bbb30a623c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:19:47Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-1233602809',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c8d2ef6638184f178a760075af39cc8b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:19:47Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:19:47Z,use_quota=True,user_id='53aa66e4f5134dcf9adb7aa238ba43de',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:47.565958 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0fc32700-17bf-45a0-bcc2-8c746caf92bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:47.570312 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0fc32700-17bf-45a0-bcc2-8c746caf92bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:47.575507 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (375e7b73-ed23-4fad-b5d9-4ac3534c692b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:47.579374 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (375e7b73-ed23-4fad-b5d9-4ac3534c692b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11', 'volume_name': 'volume-6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:47.584155 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac0a23dc-d03e-44ec-92d5-412a3a280bce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:47.587978 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac0a23dc-d03e-44ec-92d5-412a3a280bce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:47.592517 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (11270ff1-c95c-4281-9efd-260978809b75) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:47.593578 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Volume 6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11', 'volume_size': 1} Sep 17 21:19:47.593976 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] creating volume 'volume-6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:19:47.594748 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:47.654420 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:19:47.654838 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:19:47.658463 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (11270ff1-c95c-4281-9efd-260978809b75) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:47.662941 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (16b671c0-13d6-45fc-b38b-2ecd6de3aade) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:19:47.676134 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Volume volume-6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11 (6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11): created successfully Sep 17 21:19:47.680035 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (16b671c0-13d6-45fc-b38b-2ecd6de3aade) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:19:47.685366 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Flow 'volume_create_manager' (db778862-c585-455c-b967-3ccc3ee00a70) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:19:47.695888 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b6502d6d-d4cf-4f1f-9abe-4cf94802d0ae tempest-VolumesGetTest-890360817 None] Created volume successfully. Sep 17 21:19:48.642747 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] Acquiring lock "cinder-6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:48.652864 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] Lock "cinder-6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:48.695763 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:48.752186 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:48.752718 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] Volume volume-6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:48.766514 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] deleting rbd volume volume-6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:48.991770 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] Created reservations ['62ae93a5-2503-4d8b-8bf3-679e0973d7fd', '25bb4125-8c36-4659-a1d4-99f4ee798d86', '267de34c-2e36-488c-86fc-36a96dae5bc2', '059bb6cf-c56c-48ff-adf5-613573731632'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:49.039983 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:49.068064 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:49.093391 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:49.097495 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] Deleted volume successfully. Sep 17 21:19:49.110979 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-03fc36db-369a-405b-8b2b-e8acbd9e8e93 tempest-VolumesGetTest-890360817 None] Lock "cinder-6e1023f4-f3f0-49b1-bf3f-14d9f70c2d11-delete_volume" released by "delete_volume" :: held 0.458s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:49.350759 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=54,cinder:INSERT=12,cinder:UPDATE=31,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:49.361232 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-f545957f-53cc-4224-846c-66a037e5b0a9 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['4b3c625b-f346-42a8-bf16-118cf5c1e090', '552ab6de-09a6-45b5-9004-bb9cec8b2365', '2f481ac5-8c0b-41f4-98b8-74bd9d70eff4', '56c35a88-1213-462f-8fb7-5161a1d0d5df'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:49.407330 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f545957f-53cc-4224-846c-66a037e5b0a9 tempest-VolumesSnapshotTestJSON-2104722578 None] Delete snapshot completed successfully. Sep 17 21:19:49.413392 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f545957f-53cc-4224-846c-66a037e5b0a9 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-9380873c-f7d7-4333-8f51-97d1045f7951-delete_snapshot" released by "delete_snapshot" :: held 30.312s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:49.789061 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] Acquiring lock "cinder-feff5317-c25c-46fd-ab12-9e4fc570ae56-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:49.800368 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] Lock "cinder-feff5317-c25c-46fd-ab12-9e4fc570ae56-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:49.841699 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:49.908213 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:49.914412 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] deleting rbd volume volume-feff5317-c25c-46fd-ab12-9e4fc570ae56 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:50.067365 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:19:50.104041 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] deleting parent snapshot volume-feff5317-c25c-46fd-ab12-9e4fc570ae56.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:19:51.154182 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-68f183d1-310e-4019-9736-63db055ca2ba tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-01c5205d-becc-4f93-a7d8-526eb3c41bff-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:51.167600 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-68f183d1-310e-4019-9736-63db055ca2ba tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-01c5205d-becc-4f93-a7d8-526eb3c41bff-delete_snapshot" acquired by "delete_snapshot" :: waited 0.013s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:51.169097 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-68f183d1-310e-4019-9736-63db055ca2ba tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:51.268901 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] Created reservations ['3453e8cd-60f2-4d83-a549-da26b39bdc84', '79bc16dd-c61f-4787-8dd9-b3355670b032', '7bc7f7d1-ec26-453d-acc9-0da6e5f6ac88', '0e7f41fd-a5d6-40b7-877f-6b6d3b051d45'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:51.324118 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:51.348070 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:51.372346 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:51.377937 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] Deleted volume successfully. Sep 17 21:19:51.392174 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7212df6a-600b-4665-a3cb-1bc18d9af56c tempest-VolumesGetTest-890360817 None] Lock "cinder-feff5317-c25c-46fd-ab12-9e4fc570ae56-delete_volume" released by "delete_volume" :: held 1.592s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:19:51.990005 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] Acquiring lock "cinder-ac863d25-4c41-4b94-80e8-d03560d4e42e-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:19:52.009737 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] Lock "cinder-ac863d25-4c41-4b94-80e8-d03560d4e42e-delete_volume" acquired by "delete_volume" :: waited 0.020s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:19:52.047148 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:52.085428 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:19:52.085893 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] Volume volume-ac863d25-4c41-4b94-80e8-d03560d4e42e is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:19:52.091292 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] deleting rbd volume volume-ac863d25-4c41-4b94-80e8-d03560d4e42e {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:19:52.247979 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=26,cinder:SELECT=57,cinder:INSERT=14,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:19:52.277045 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] Created reservations ['1399479f-928d-4ab9-a1c5-adf1638c37c8', 'a6713780-c7c3-4494-9f23-7fe696c224dd', '6624e7bc-117a-4773-92de-eca2caa8dfce', '6ca91a1e-27f3-48fe-9e66-a50561992fc1'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:19:52.337157 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:52.379396 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:19:52.407665 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:19:52.411933 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] Deleted volume successfully. Sep 17 21:19:52.425578 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f810b8ef-1746-4750-ab7b-ca346c3885f7 tempest-VolumesGetTest-890360817 None] Lock "cinder-ac863d25-4c41-4b94-80e8-d03560d4e42e-delete_volume" released by "delete_volume" :: held 0.416s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:20:01.032099 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:20:01.033990 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:01.098787 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:01.183334 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:20:03.292315 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=34,cinder:UPDATE=20,cinder:INSERT=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:03.385693 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:20:03.433544 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Flow 'volume_create_manager' (e33eec14-7689-449b-8dd4-5a96beaf6edf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:20:03.447099 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1c04e4bf-0fb7-4aa7-955a-39c3d3b77872) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:03.466860 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1c04e4bf-0fb7-4aa7-955a-39c3d3b77872) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:20:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-446381109',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2b705eb5-8de6-4864-836d-ce99f20a165c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8d3fab3adf0c425680c91af8afcc516f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:20:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:20:03Z,use_quota=True,user_id='7ff5b886208a44c0891c2d1ba65dc083',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:03.475252 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c59f3c87-b555-4498-be56-e8bd17541a7d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:03.480495 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c59f3c87-b555-4498-be56-e8bd17541a7d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:03.486051 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c32509c9-b66b-46ad-b64a-e5d29654ade7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:03.490305 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c32509c9-b66b-46ad-b64a-e5d29654ade7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2b705eb5-8de6-4864-836d-ce99f20a165c', 'volume_name': 'volume-2b705eb5-8de6-4864-836d-ce99f20a165c', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:03.495790 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (aa9fc99d-93c0-455f-bb40-a0f37b1eb502) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:03.499800 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (aa9fc99d-93c0-455f-bb40-a0f37b1eb502) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:03.505034 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (24f628df-1423-419c-9f13-281f78c15c0f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:03.506152 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Volume 2b705eb5-8de6-4864-836d-ce99f20a165c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2b705eb5-8de6-4864-836d-ce99f20a165c', 'volume_size': 1} Sep 17 21:20:03.506596 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] creating volume 'volume-2b705eb5-8de6-4864-836d-ce99f20a165c' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:20:03.507634 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:03.579656 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:20:03.579656 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:20:03.583673 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (24f628df-1423-419c-9f13-281f78c15c0f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2b705eb5-8de6-4864-836d-ce99f20a165c', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:03.588626 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a903a647-3d44-49f2-8b02-5ca7ac50c19a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:03.604464 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Volume volume-2b705eb5-8de6-4864-836d-ce99f20a165c (2b705eb5-8de6-4864-836d-ce99f20a165c): created successfully Sep 17 21:20:03.610891 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a903a647-3d44-49f2-8b02-5ca7ac50c19a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:03.618802 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Flow 'volume_create_manager' (e33eec14-7689-449b-8dd4-5a96beaf6edf) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:20:03.633439 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7ab2147d-6551-4582-a542-69e8ef285025 tempest-AttachVolumeTestJSON-890686982 None] Created volume successfully. Sep 17 21:20:05.259663 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-1b174f0e-05e2-48c6-b131-cb2c94fed909 req-10f86cc0-ee0b-48ff-b0f6-bda1afbae637 tempest-AttachVolumeTestJSON-890686982 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:20:06.199980 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-1b174f0e-05e2-48c6-b131-cb2c94fed909 req-10f86cc0-ee0b-48ff-b0f6-bda1afbae637 tempest-AttachVolumeTestJSON-890686982 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.940s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:20:06.200569 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-1b174f0e-05e2-48c6-b131-cb2c94fed909 req-10f86cc0-ee0b-48ff-b0f6-bda1afbae637 tempest-AttachVolumeTestJSON-890686982 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2b705eb5-8de6-4864-836d-ce99f20a165c', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '2b705eb5-8de6-4864-836d-ce99f20a165c', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:20:06.238175 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1b174f0e-05e2-48c6-b131-cb2c94fed909 req-10f86cc0-ee0b-48ff-b0f6-bda1afbae637 tempest-AttachVolumeTestJSON-890686982 None] Connection info returned from driver {'name': 'volumes/volume-2b705eb5-8de6-4864-836d-ce99f20a165c', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '2b705eb5-8de6-4864-836d-ce99f20a165c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '0bb5744d-bb9a-45b7-8af6-acebde574374'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:20:06.311138 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-1b174f0e-05e2-48c6-b131-cb2c94fed909 req-10f86cc0-ee0b-48ff-b0f6-bda1afbae637 tempest-AttachVolumeTestJSON-890686982 None] attachment_update completed successfully. Sep 17 21:20:09.192845 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:20:09.215167 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Flow 'volume_create_manager' (faf57dcd-2780-43ad-90be-f746392ed799) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:20:09.222026 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (46705615-065f-4756-a195-fc8523f26347) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:09.253667 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (46705615-065f-4756-a195-fc8523f26347) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:20:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-1852981955',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=97fd9efa-2583-43e8-ae8f-bced2696518e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8d3fab3adf0c425680c91af8afcc516f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:20:09Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:20:09Z,use_quota=True,user_id='7ff5b886208a44c0891c2d1ba65dc083',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:09.269449 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1132ed17-4934-496d-a23a-d54cd613e54f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:09.274100 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1132ed17-4934-496d-a23a-d54cd613e54f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:09.298564 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c9514a33-2b65-4ba7-aca1-44bf751ef069) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:09.305507 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c9514a33-2b65-4ba7-aca1-44bf751ef069) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '97fd9efa-2583-43e8-ae8f-bced2696518e', 'volume_name': 'volume-97fd9efa-2583-43e8-ae8f-bced2696518e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:09.348961 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bf468be9-d0b8-48bb-9b50-d17167e7d0ad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:09.360303 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bf468be9-d0b8-48bb-9b50-d17167e7d0ad) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:09.367060 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bf8f96cd-4d75-411c-b1c3-a1592fcd61d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:09.368763 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Volume 97fd9efa-2583-43e8-ae8f-bced2696518e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-97fd9efa-2583-43e8-ae8f-bced2696518e', 'volume_size': 1} Sep 17 21:20:09.369381 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] creating volume 'volume-97fd9efa-2583-43e8-ae8f-bced2696518e' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:20:09.370286 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:09.462876 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:20:09.464785 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:20:09.471146 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bf8f96cd-4d75-411c-b1c3-a1592fcd61d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-97fd9efa-2583-43e8-ae8f-bced2696518e', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:09.477568 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2030c588-7c5b-449f-a987-d2102f5e3206) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:09.496053 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Volume volume-97fd9efa-2583-43e8-ae8f-bced2696518e (97fd9efa-2583-43e8-ae8f-bced2696518e): created successfully Sep 17 21:20:09.501557 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2030c588-7c5b-449f-a987-d2102f5e3206) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:09.508722 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Flow 'volume_create_manager' (faf57dcd-2780-43ad-90be-f746392ed799) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:20:09.521167 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3590c124-75dd-4e8c-adc7-3d8831930a05 tempest-AttachVolumeTestJSON-890686982 None] Created volume successfully. Sep 17 21:20:10.768430 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-066a6211-5b2f-42c1-99b1-fc260c3b14b5 req-cc44639d-ab9f-4f1d-a460-b44498c6952e tempest-AttachVolumeTestJSON-890686982 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:20:11.052104 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] Acquiring lock "cinder-e7b28ecd-7573-4b77-a5cd-36f4afa22ac4-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:20:11.070938 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] Lock "cinder-e7b28ecd-7573-4b77-a5cd-36f4afa22ac4-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:20:11.125888 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:11.177476 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:20:11.177942 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] Volume volume-e7b28ecd-7573-4b77-a5cd-36f4afa22ac4 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:20:11.182562 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] deleting rbd volume volume-e7b28ecd-7573-4b77-a5cd-36f4afa22ac4 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:20:11.394028 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] Created reservations ['da84599d-4731-47c4-944e-9c594e91a3f3', 'b9871a05-7999-4da1-b8c7-132b50b5e041', '2badeb2a-030a-4d68-9a84-afd7d81bd073', 'ce4d4a93-2221-4377-a130-786189b5f318'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:20:11.452383 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:11.485395 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:11.525000 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:20:11.542020 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] Deleted volume successfully. Sep 17 21:20:11.561987 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f9379815-b7f2-4b64-849f-e80c305128ed tempest-AttachVolumeShelveTestJSON-1971975439 None] Lock "cinder-e7b28ecd-7573-4b77-a5cd-36f4afa22ac4-delete_volume" released by "delete_volume" :: held 0.491s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:20:11.618897 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-066a6211-5b2f-42c1-99b1-fc260c3b14b5 req-cc44639d-ab9f-4f1d-a460-b44498c6952e tempest-AttachVolumeTestJSON-890686982 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.850s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:20:11.619542 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-066a6211-5b2f-42c1-99b1-fc260c3b14b5 req-cc44639d-ab9f-4f1d-a460-b44498c6952e tempest-AttachVolumeTestJSON-890686982 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-97fd9efa-2583-43e8-ae8f-bced2696518e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '97fd9efa-2583-43e8-ae8f-bced2696518e', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:20:11.645563 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-066a6211-5b2f-42c1-99b1-fc260c3b14b5 req-cc44639d-ab9f-4f1d-a460-b44498c6952e tempest-AttachVolumeTestJSON-890686982 None] Connection info returned from driver {'name': 'volumes/volume-97fd9efa-2583-43e8-ae8f-bced2696518e', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '97fd9efa-2583-43e8-ae8f-bced2696518e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '6f853864-70d0-4b6e-87c5-ecb390f85174'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:20:11.715710 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-066a6211-5b2f-42c1-99b1-fc260c3b14b5 req-cc44639d-ab9f-4f1d-a460-b44498c6952e tempest-AttachVolumeTestJSON-890686982 None] attachment_update completed successfully. Sep 17 21:20:12.311650 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] Acquiring lock "cinder-b2bc9cfa-f40b-411e-8a44-20b3f6134615-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:20:12.323977 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] Lock "cinder-b2bc9cfa-f40b-411e-8a44-20b3f6134615-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:20:12.357296 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:12.402264 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:20:12.402724 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] Volume volume-b2bc9cfa-f40b-411e-8a44-20b3f6134615 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:20:12.406977 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] deleting rbd volume volume-b2bc9cfa-f40b-411e-8a44-20b3f6134615 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:20:12.597907 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] Created reservations ['92511f17-706e-46c3-acf2-dfec1c49c5df', '3c76acf1-792c-43c7-9254-ad4c3150a1f7', 'c6090c51-61bd-45e7-8886-b3eb9770d874', '2d339671-39dc-4df0-bbff-ed7fed3460e3'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:20:12.635371 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:INSERT=7,cinder:SELECT=54,cinder:UPDATE=36,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:12.645978 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:12.693406 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:12.724269 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:20:12.728669 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] Deleted volume successfully. Sep 17 21:20:12.753516 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2d2378b7-6172-4254-8323-3fe603a6c8d6 tempest-AttachVolumeShelveTestJSON-1971975439 None] Lock "cinder-b2bc9cfa-f40b-411e-8a44-20b3f6134615-delete_volume" released by "delete_volume" :: held 0.429s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:20:14.988392 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-e4318ef5-04dc-4155-818b-6ab1dfc92bdc req-ae3d62bc-802e-4470-b080-4c1c32857764 tempest-AttachVolumeTestJSON-890686982 None] Terminate volume connection completed successfully. Sep 17 21:20:14.988949 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e4318ef5-04dc-4155-818b-6ab1dfc92bdc req-ae3d62bc-802e-4470-b080-4c1c32857764 tempest-AttachVolumeTestJSON-890686982 None] Deleting attachment 0bb5744d-bb9a-45b7-8af6-acebde574374. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:20:16.496483 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-56b446ad-7eff-4d5e-b092-a9738461f745 req-515b747b-6a2a-43b0-ab76-e039b874d6a6 tempest-AttachVolumeTestJSON-890686982 None] Terminate volume connection completed successfully. Sep 17 21:20:16.497568 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56b446ad-7eff-4d5e-b092-a9738461f745 req-515b747b-6a2a-43b0-ab76-e039b874d6a6 tempest-AttachVolumeTestJSON-890686982 None] Deleting attachment 6f853864-70d0-4b6e-87c5-ecb390f85174. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:20:21.363619 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-68f183d1-310e-4019-9736-63db055ca2ba tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['6e5dbd1d-ecff-47cd-aa76-96191bcea762', 'f1875d87-46ef-4adc-a614-e56e7dfd0aa8', 'ad990c06-0a25-4e9d-b832-aa4396ec4540', '5da0c004-043e-4571-8ab1-68ab374e1f27'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:20:21.411493 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=72,cinder:UPDATE=22,cinder:DELETE=1,cinder:INSERT=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:21.428234 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-68f183d1-310e-4019-9736-63db055ca2ba tempest-VolumesSnapshotTestJSON-2104722578 None] Delete snapshot completed successfully. Sep 17 21:20:21.434687 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-68f183d1-310e-4019-9736-63db055ca2ba tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-01c5205d-becc-4f93-a7d8-526eb3c41bff-delete_snapshot" released by "delete_snapshot" :: held 30.267s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:20:21.565988 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-812723a2-4ca4-4d80-b127-0ef039ef85b6 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-56123665-b21d-4895-8cdd-43833775a67f-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:20:21.584931 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-812723a2-4ca4-4d80-b127-0ef039ef85b6 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-56123665-b21d-4895-8cdd-43833775a67f-delete_snapshot" acquired by "delete_snapshot" :: waited 0.019s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:20:21.587297 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-812723a2-4ca4-4d80-b127-0ef039ef85b6 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:22.105757 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:20:22.131898 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Flow 'volume_create_manager' (e1084044-0eb1-48a4-b24f-4dea8521b7c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:20:22.139545 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (60a713b0-3445-49b4-a7a3-2dd82653c8cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:22.157560 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (60a713b0-3445-49b4-a7a3-2dd82653c8cc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:20:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotsTestJSON-volume-2141890098',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f2fd0343346a41008083043c91e87aab',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:20:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:20:22Z,use_quota=True,user_id='b3f951e315234c69b0c1294ddac832ef',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:22.163953 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (294374a6-c759-4cc4-a0de-f9378e0cf61c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:22.175189 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (294374a6-c759-4cc4-a0de-f9378e0cf61c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:22.181629 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3e9966a8-9b8b-4b95-b1e0-d6b833959cfe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:22.186998 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3e9966a8-9b8b-4b95-b1e0-d6b833959cfe) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b', 'volume_name': 'volume-90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:22.193857 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (582416a9-99f0-4861-b6bd-a8b5163f0537) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:22.198729 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (582416a9-99f0-4861-b6bd-a8b5163f0537) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:22.204910 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5d795ab2-fadb-4175-b3dc-ba13356ef795) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:22.206390 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Volume 90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b', 'volume_size': 1} Sep 17 21:20:22.207294 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] creating volume 'volume-90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:20:22.208554 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:22.327745 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:20:22.328307 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:20:22.332423 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5d795ab2-fadb-4175-b3dc-ba13356ef795) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:22.341398 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (055623f8-630c-4965-b942-6bc9a60e08f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:20:22.376740 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Volume volume-90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b (90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b): created successfully Sep 17 21:20:22.382071 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (055623f8-630c-4965-b942-6bc9a60e08f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:20:22.390719 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Flow 'volume_create_manager' (e1084044-0eb1-48a4-b24f-4dea8521b7c0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:20:22.414098 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-e73b181f-b327-4c7a-93e1-a98d7ac2e15d req-00cb37e1-5c3c-4ad7-bf38-a5846d106397 tempest-VolumesSnapshotsTestJSON-474974175 None] Created volume successfully. Sep 17 21:20:23.212518 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] Acquiring lock "cinder-97fd9efa-2583-43e8-ae8f-bced2696518e-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:20:23.225603 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] Lock "cinder-97fd9efa-2583-43e8-ae8f-bced2696518e-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:20:23.274918 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:23.361783 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:20:23.362348 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] Volume volume-97fd9efa-2583-43e8-ae8f-bced2696518e is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:20:23.371724 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] deleting rbd volume volume-97fd9efa-2583-43e8-ae8f-bced2696518e {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:20:23.514221 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-66db7bd9-7ff2-41b3-8cd8-2ea0110a1fbb req-fe985297-6b01-4b8d-a950-cc7416f7f225 tempest-VolumesSnapshotsTestJSON-474974175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:23.753928 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] Created reservations ['f77c07f5-d289-47fe-b043-8a1f7804ccfd', '47c772f1-8ba9-405c-831c-6663ec93071a', 'b6d006d8-3568-4e83-89f5-8f386ff81a8b', '6ee929bc-9d3b-49d8-adaa-2c1fd5cb3194'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:20:23.799801 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:23.829063 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:23.857296 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:20:23.862451 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] Deleted volume successfully. Sep 17 21:20:23.887867 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-58ac44d7-0c8e-4973-ae67-8c9f38d8a2f5 tempest-AttachVolumeTestJSON-890686982 None] Lock "cinder-97fd9efa-2583-43e8-ae8f-bced2696518e-delete_volume" released by "delete_volume" :: held 0.662s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:20:24.402741 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] Acquiring lock "cinder-2b705eb5-8de6-4864-836d-ce99f20a165c-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:20:24.417921 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] Lock "cinder-2b705eb5-8de6-4864-836d-ce99f20a165c-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:20:24.452301 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:24.509403 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:20:24.510055 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] Volume volume-2b705eb5-8de6-4864-836d-ce99f20a165c is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:20:24.515229 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] deleting rbd volume volume-2b705eb5-8de6-4864-836d-ce99f20a165c {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:20:24.570314 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-66db7bd9-7ff2-41b3-8cd8-2ea0110a1fbb req-fe985297-6b01-4b8d-a950-cc7416f7f225 tempest-VolumesSnapshotsTestJSON-474974175 None] Create snapshot completed successfully Sep 17 21:20:24.736009 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] Created reservations ['407c8f63-c558-4d8f-8afb-1cbe1a088829', '553440fa-6727-4924-922c-377cc3951a40', '30f57bfc-9740-4bc0-b62c-a595b640cd52', '46521b00-1f59-4ea2-aed0-967790770a11'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:20:24.800085 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:24.849994 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-b2d5c7f1-fdbe-40c7-a4b4-498a5b382926 req-06300675-b1fe-40a7-a8ac-943b1d7d55f4 tempest-VolumesSnapshotsTestJSON-474974175 None] Acquiring lock "cinder-175be0c3-8086-4664-b498-1d003eda5f30-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:20:24.853717 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:24.891713 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:20:24.897741 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] Deleted volume successfully. Sep 17 21:20:24.909201 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-b2d5c7f1-fdbe-40c7-a4b4-498a5b382926 req-06300675-b1fe-40a7-a8ac-943b1d7d55f4 tempest-VolumesSnapshotsTestJSON-474974175 None] Lock "cinder-175be0c3-8086-4664-b498-1d003eda5f30-delete_snapshot" acquired by "delete_snapshot" :: waited 0.059s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:20:24.910890 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-b2d5c7f1-fdbe-40c7-a4b4-498a5b382926 req-06300675-b1fe-40a7-a8ac-943b1d7d55f4 tempest-VolumesSnapshotsTestJSON-474974175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:24.957218 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-28b1cdb7-3c1c-492d-8f7e-63d479a9663c tempest-AttachVolumeTestJSON-890686982 None] Lock "cinder-2b705eb5-8de6-4864-836d-ce99f20a165c-delete_volume" released by "delete_volume" :: held 0.539s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:20:25.609938 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [req-b2d5c7f1-fdbe-40c7-a4b4-498a5b382926 req-06300675-b1fe-40a7-a8ac-943b1d7d55f4 tempest-VolumesSnapshotsTestJSON-474974175 None] Created reservations ['77b3ec69-2474-4da5-a74c-db8b92513b20', '92c88a05-d1f4-4137-bbad-60656840cf1c', 'f49416bb-6e6e-4b86-b5a6-392ad2c1b687', '9aa5560d-64e2-45dc-8c0f-6855f4404b7e'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:20:25.614919 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=56,cinder:DELETE=3,cinder:UPDATE=32,cinder:INSERT=9 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:25.624425 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] Acquiring lock "cinder-779942d3-ce89-44ed-bbfc-e00719495d8e-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:20:25.678374 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] Lock "cinder-779942d3-ce89-44ed-bbfc-e00719495d8e-delete_volume" acquired by "delete_volume" :: waited 0.054s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:20:25.719772 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-b2d5c7f1-fdbe-40c7-a4b4-498a5b382926 req-06300675-b1fe-40a7-a8ac-943b1d7d55f4 tempest-VolumesSnapshotsTestJSON-474974175 None] Delete snapshot completed successfully. Sep 17 21:20:25.725024 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:25.799664 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:20:25.799664 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] Volume volume-779942d3-ce89-44ed-bbfc-e00719495d8e is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:20:25.811205 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] deleting rbd volume volume-779942d3-ce89-44ed-bbfc-e00719495d8e {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:20:25.813153 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-b2d5c7f1-fdbe-40c7-a4b4-498a5b382926 req-06300675-b1fe-40a7-a8ac-943b1d7d55f4 tempest-VolumesSnapshotsTestJSON-474974175 None] Lock "cinder-175be0c3-8086-4664-b498-1d003eda5f30-delete_snapshot" released by "delete_snapshot" :: held 0.904s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:20:26.007188 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] Acquiring lock "cinder-90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:20:26.023080 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] Lock "cinder-90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:20:26.056865 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] Created reservations ['ce1cebe9-756a-4cbf-9483-39bf6c04e710', 'e9e3a8c2-d7a5-45fd-8e00-4620e49d5840', '80856d7a-cd0f-412f-949c-e0c4f909c3b4', 'f581d456-3a7c-403c-9929-49a33a90138d'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:20:26.092204 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:26.124888 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:20:26.125486 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] Volume volume-90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:20:26.130591 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] deleting rbd volume volume-90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:20:26.148642 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=63,cinder:UPDATE=29,cinder:INSERT=6,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:26.164663 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:26.191800 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:26.219864 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:20:26.224672 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] Deleted volume successfully. Sep 17 21:20:26.239364 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-4937c266-30b1-4dcf-b900-4cb2ad6b44d8 tempest-AttachVolumeTestJSON-890686982 None] Lock "cinder-779942d3-ce89-44ed-bbfc-e00719495d8e-delete_volume" released by "delete_volume" :: held 0.561s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:20:26.334193 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] Created reservations ['6abb826c-3b55-4bba-8732-0e4c2166e355', '1faf255b-56f8-415d-9b40-26ae982491be', 'b2bc331f-347e-45a1-917d-db08ed5d336e', '32ee5eb6-3490-4e7b-a4c1-27d2620e74ca'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:20:26.379574 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:26.417677 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:26.445010 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:20:26.449073 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] Deleted volume successfully. Sep 17 21:20:26.462101 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-038fd41a-a27f-4a98-af0d-53aab19b0e3e req-e99249ff-909a-4d24-8c01-6453a2e52112 tempest-VolumesSnapshotsTestJSON-474974175 None] Lock "cinder-90a7bd6c-2c29-45f5-a97f-cfde9a3fd96b-delete_volume" released by "delete_volume" :: held 0.439s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:20:43.389287 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=9,cinder:INSERT=3,cinder:UPDATE=7 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:51.859713 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-812723a2-4ca4-4d80-b127-0ef039ef85b6 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['1de23360-c238-4813-a22f-ad3d65ba146a', 'ef0f2866-d253-4425-83ca-7f88e92a280b', 'cf47125a-c495-43a9-b5f8-838c7f2f7c80', '1766aef8-83f7-411d-a1a5-220ff1d75b47'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:20:51.916680 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-812723a2-4ca4-4d80-b127-0ef039ef85b6 tempest-VolumesSnapshotTestJSON-2104722578 None] Delete snapshot completed successfully. Sep 17 21:20:51.922699 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-812723a2-4ca4-4d80-b127-0ef039ef85b6 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-56123665-b21d-4895-8cdd-43833775a67f-delete_snapshot" released by "delete_snapshot" :: held 30.338s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:20:53.459186 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-9effa8ab-137e-4166-a17c-48e8a1483b77 req-2572914c-9fc5-446a-bd0d-4600d911061f tempest-VolumesSnapshotTestJSON-2104722578 None] Terminate volume connection completed successfully. Sep 17 21:20:53.459880 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-9effa8ab-137e-4166-a17c-48e8a1483b77 req-2572914c-9fc5-446a-bd0d-4600d911061f tempest-VolumesSnapshotTestJSON-2104722578 None] Deleting attachment 73094c43-5e1a-43c9-ab80-7e98a3223b91. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:20:59.153623 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=28,cinder:SELECT=43,cinder:INSERT=7,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:20:59.170653 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-54e05f35-bcbc-4750-b61f-9fb340d9817f tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:20:59.580084 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-54e05f35-bcbc-4750-b61f-9fb340d9817f tempest-VolumesSnapshotTestJSON-2104722578 None] Create snapshot completed successfully Sep 17 21:21:00.499684 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-aa0aca3c-4ad4-4611-83dc-bec45c0bd625 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-384b026c-d1f7-4bbf-bae5-1520bbf6cb3a-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:21:00.522257 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-aa0aca3c-4ad4-4611-83dc-bec45c0bd625 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-384b026c-d1f7-4bbf-bae5-1520bbf6cb3a-delete_snapshot" acquired by "delete_snapshot" :: waited 0.023s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:21:00.529139 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-aa0aca3c-4ad4-4611-83dc-bec45c0bd625 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:01.028054 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:21:01.029207 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:01.088971 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:01.166586 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:21:01.659115 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-aa0aca3c-4ad4-4611-83dc-bec45c0bd625 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['184cb0a4-1288-4e2f-8de6-570afa7f8021', '0f4e447e-6aeb-4658-bf1d-a330a292005f', '43f1fd5e-5946-4bf1-902c-734ea0eedd2e', '08b9ed42-e85f-4f1d-b1c8-506fa18cee6a'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:21:01.762294 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-aa0aca3c-4ad4-4611-83dc-bec45c0bd625 tempest-VolumesSnapshotTestJSON-2104722578 None] Delete snapshot completed successfully. Sep 17 21:21:01.778233 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-aa0aca3c-4ad4-4611-83dc-bec45c0bd625 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-384b026c-d1f7-4bbf-bae5-1520bbf6cb3a-delete_snapshot" released by "delete_snapshot" :: held 1.256s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:21:02.901106 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8e548038-cfab-4ac3-9e35-e5c3229b919c tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:03.627408 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8e548038-cfab-4ac3-9e35-e5c3229b919c tempest-VolumesSnapshotTestJSON-2104722578 None] Create snapshot completed successfully Sep 17 21:21:18.803015 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=33,cinder:INSERT=4,cinder:UPDATE=20,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:18.804850 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-ad2484ce-9e3d-413d-8d16-d682916b076d req-929c406f-c051-4387-81d7-b078a0b7b91e tempest-VolumesSnapshotTestJSON-2104722578 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:21:19.929561 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-ad2484ce-9e3d-413d-8d16-d682916b076d req-929c406f-c051-4387-81d7-b078a0b7b91e tempest-VolumesSnapshotTestJSON-2104722578 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.123s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:21:19.932501 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-ad2484ce-9e3d-413d-8d16-d682916b076d req-929c406f-c051-4387-81d7-b078a0b7b91e tempest-VolumesSnapshotTestJSON-2104722578 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-5b712483-f63f-4381-9968-4e3fa55f72b6', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '5b712483-f63f-4381-9968-4e3fa55f72b6', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:21:19.964823 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-ad2484ce-9e3d-413d-8d16-d682916b076d req-929c406f-c051-4387-81d7-b078a0b7b91e tempest-VolumesSnapshotTestJSON-2104722578 None] Connection info returned from driver {'name': 'volumes/volume-5b712483-f63f-4381-9968-4e3fa55f72b6', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '5b712483-f63f-4381-9968-4e3fa55f72b6', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'aa7ef21b-4604-40f7-ad32-2060f4898ef1'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:21:20.039178 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-ad2484ce-9e3d-413d-8d16-d682916b076d req-929c406f-c051-4387-81d7-b078a0b7b91e tempest-VolumesSnapshotTestJSON-2104722578 None] attachment_update completed successfully. Sep 17 21:21:22.387241 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2b2e3cb4-3ad1-48bf-adfb-89bc8af34f3b tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:23.725846 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2b2e3cb4-3ad1-48bf-adfb-89bc8af34f3b tempest-VolumesSnapshotTestJSON-2104722578 None] Create snapshot completed successfully Sep 17 21:21:24.950005 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5690b368-1c31-41cf-b0b9-dc64df275c0d tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:25.737182 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5690b368-1c31-41cf-b0b9-dc64df275c0d tempest-VolumesSnapshotTestJSON-2104722578 None] Create snapshot completed successfully Sep 17 21:21:26.040484 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-168a0453-6d03-4679-8414-fc7bc1b2edee tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-11c40bf8-6838-4745-8fdf-f201c83bbbc2-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:21:26.086771 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-168a0453-6d03-4679-8414-fc7bc1b2edee tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-11c40bf8-6838-4745-8fdf-f201c83bbbc2-delete_snapshot" acquired by "delete_snapshot" :: waited 0.046s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:21:26.093792 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-168a0453-6d03-4679-8414-fc7bc1b2edee tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:33.389959 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=38,cinder:UPDATE=7,cinder:INSERT=1,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:43.053184 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b7f86695-fd49-49f4-ba6f-d4bbf99cb99d tempest-GroupsV320Test-1802824746 None] Group tempest-GroupsV320Test-Group-182109489: creating Sep 17 21:21:43.080178 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b7f86695-fd49-49f4-ba6f-d4bbf99cb99d tempest-GroupsV320Test-1802824746 None] Group tempest-GroupsV320Test-Group-182109489: created successfully Sep 17 21:21:43.092587 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b7f86695-fd49-49f4-ba6f-d4bbf99cb99d tempest-GroupsV320Test-1802824746 None] Create group completed successfully. Sep 17 21:21:44.813153 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-594c3ce4-318b-4d9a-a049-f1b90aca9a52 tempest-GroupsV320Test-1802824746 None] Created reservations ['a3b72b4f-8dfb-482a-ac6c-9f0823082a83'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:21:44.878444 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-594c3ce4-318b-4d9a-a049-f1b90aca9a52 tempest-GroupsV320Test-1802824746 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:44.927151 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-594c3ce4-318b-4d9a-a049-f1b90aca9a52 tempest-GroupsV320Test-1802824746 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:45.010182 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-594c3ce4-318b-4d9a-a049-f1b90aca9a52 tempest-GroupsV320Test-1802824746 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:21:45.017980 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-594c3ce4-318b-4d9a-a049-f1b90aca9a52 tempest-GroupsV320Test-1802824746 None] Delete group completed successfully. Sep 17 21:21:51.176453 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=48,cinder:UPDATE=12,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:21:51.206776 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:21:51.231439 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Flow 'volume_create_manager' (4c8b0502-e839-4f5a-802a-c7bf939e1e63) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:21:51.239652 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1610bfea-adc9-440b-91f1-9d3c1be2332f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:21:51.259510 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1610bfea-adc9-440b-91f1-9d3c1be2332f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:21:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotMetadataTestJSON-Volume-732988094',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5b1fd599-aed4-48e2-bc68-47d51f96e8fa,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d37289e9d7394e5b86c0b7c10f97b365',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:21:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:21:51Z,use_quota=True,user_id='d46bf9c0b90f4589989b468cb4feec28',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:21:51.266529 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (36f5fae9-91f4-4b4a-9343-e7158678ff37) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:21:51.272195 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (36f5fae9-91f4-4b4a-9343-e7158678ff37) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:21:51.277369 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4aaa9417-364d-42fc-9872-83b7b2dd37fa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:21:51.283257 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4aaa9417-364d-42fc-9872-83b7b2dd37fa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5b1fd599-aed4-48e2-bc68-47d51f96e8fa', 'volume_name': 'volume-5b1fd599-aed4-48e2-bc68-47d51f96e8fa', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:21:51.288535 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5a8234cb-bb36-4669-8c73-0fb6c6f207c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:21:51.293145 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5a8234cb-bb36-4669-8c73-0fb6c6f207c5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:21:51.298411 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7ea5069b-a843-4771-a711-b9a568ba7089) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:21:51.300175 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Volume 5b1fd599-aed4-48e2-bc68-47d51f96e8fa: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5b1fd599-aed4-48e2-bc68-47d51f96e8fa', 'volume_size': 1} Sep 17 21:21:51.300738 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] creating volume 'volume-5b1fd599-aed4-48e2-bc68-47d51f96e8fa' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:21:51.301709 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:51.400919 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:21:51.401579 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:21:51.405581 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7ea5069b-a843-4771-a711-b9a568ba7089) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5b1fd599-aed4-48e2-bc68-47d51f96e8fa', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:21:51.411804 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (82026fee-ac97-4ce6-ab54-e5cd1a1efdd7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:21:51.425191 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Volume volume-5b1fd599-aed4-48e2-bc68-47d51f96e8fa (5b1fd599-aed4-48e2-bc68-47d51f96e8fa): created successfully Sep 17 21:21:51.432003 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (82026fee-ac97-4ce6-ab54-e5cd1a1efdd7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:21:51.439018 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Flow 'volume_create_manager' (4c8b0502-e839-4f5a-802a-c7bf939e1e63) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:21:51.459894 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-95a0ebfd-f693-4060-890b-fb3cd66b9cf5 tempest-SnapshotMetadataTestJSON-1205306100 None] Created volume successfully. Sep 17 21:21:52.535371 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ae614a88-87ae-4784-9b4b-35942f0cb8dc tempest-SnapshotMetadataTestJSON-1205306100 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:53.644212 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ae614a88-87ae-4784-9b4b-35942f0cb8dc tempest-SnapshotMetadataTestJSON-1205306100 None] Create snapshot completed successfully Sep 17 21:21:55.251160 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7ca5d56b-30ff-40ab-a1db-e6080c7d81a4 tempest-SnapshotMetadataTestJSON-1205306100 None] Acquiring lock "cinder-78e0ab4e-8d94-492b-b22b-8d237bd19545-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:21:55.267420 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7ca5d56b-30ff-40ab-a1db-e6080c7d81a4 tempest-SnapshotMetadataTestJSON-1205306100 None] Lock "cinder-78e0ab4e-8d94-492b-b22b-8d237bd19545-delete_snapshot" acquired by "delete_snapshot" :: waited 0.017s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:21:55.269135 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7ca5d56b-30ff-40ab-a1db-e6080c7d81a4 tempest-SnapshotMetadataTestJSON-1205306100 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:56.324364 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-168a0453-6d03-4679-8414-fc7bc1b2edee tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['90a47cc4-652a-45a6-8f44-e9abd1643fab', '2c2df372-4d95-4ebc-a171-2cfe67e00e9c', '091f7e6f-fc0c-4ecb-a649-ab70425f391f', 'affd9bfe-d367-4924-b779-7b9b842cac20'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:21:56.404655 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-168a0453-6d03-4679-8414-fc7bc1b2edee tempest-VolumesSnapshotTestJSON-2104722578 None] Delete snapshot completed successfully. Sep 17 21:21:56.412847 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-168a0453-6d03-4679-8414-fc7bc1b2edee tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-11c40bf8-6838-4745-8fdf-f201c83bbbc2-delete_snapshot" released by "delete_snapshot" :: held 30.326s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:21:56.452144 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-7ca5d56b-30ff-40ab-a1db-e6080c7d81a4 tempest-SnapshotMetadataTestJSON-1205306100 None] Created reservations ['d6bbc89b-8d57-4604-b76c-c098165dad93', '6eb19697-9050-40cc-8d07-e0db9222212b', 'b77c85cc-5881-4258-8471-6bf82acc26b7', 'cbf701e1-ecfb-47cd-9bf5-81cad8c383d2'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:21:56.507792 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d2cde9ad-b9f5-49a9-b212-88c96842183f tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-6e2b57cf-090b-4f00-9538-d73a42008839-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:21:56.548914 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d2cde9ad-b9f5-49a9-b212-88c96842183f tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-6e2b57cf-090b-4f00-9538-d73a42008839-delete_snapshot" acquired by "delete_snapshot" :: waited 0.041s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:21:56.551206 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d2cde9ad-b9f5-49a9-b212-88c96842183f tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:56.648477 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7ca5d56b-30ff-40ab-a1db-e6080c7d81a4 tempest-SnapshotMetadataTestJSON-1205306100 None] Delete snapshot completed successfully. Sep 17 21:21:56.654166 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-7ca5d56b-30ff-40ab-a1db-e6080c7d81a4 tempest-SnapshotMetadataTestJSON-1205306100 None] Lock "cinder-78e0ab4e-8d94-492b-b22b-8d237bd19545-delete_snapshot" released by "delete_snapshot" :: held 1.387s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:21:57.473402 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] Acquiring lock "cinder-5b1fd599-aed4-48e2-bc68-47d51f96e8fa-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:21:57.489209 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] Lock "cinder-5b1fd599-aed4-48e2-bc68-47d51f96e8fa-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:21:57.559233 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:57.654228 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:21:57.654737 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] Volume volume-5b1fd599-aed4-48e2-bc68-47d51f96e8fa is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:21:57.668622 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] deleting rbd volume volume-5b1fd599-aed4-48e2-bc68-47d51f96e8fa {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:21:57.921166 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] Created reservations ['86e73462-ccdd-4c61-bcda-22523163d199', '6e2258df-e84a-4f3e-8283-fab296a1f0b8', 'f679cd22-3faa-40c3-9d73-7dcab2dc0230', 'e2b46f1f-43e6-4903-8f55-6ffc12ec16f7'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:21:58.010198 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:58.065256 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:21:58.113089 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:21:58.122009 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] Deleted volume successfully. Sep 17 21:21:58.136174 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-fc416b24-6b87-42a1-9ee7-c1246e803df8 tempest-SnapshotMetadataTestJSON-1205306100 None] Lock "cinder-5b1fd599-aed4-48e2-bc68-47d51f96e8fa-delete_volume" released by "delete_volume" :: held 0.647s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:22:01.032460 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:22:01.033986 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:01.091674 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:01.155710 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:22:01.770279 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:22:01.807005 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Flow 'volume_create_manager' (2c74cae1-911e-4596-9fff-d73bf7760bf9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:22:01.818602 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a1109a72-2ea8-4b2c-aeda-73fb70259c5b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:01.847492 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a1109a72-2ea8-4b2c-aeda-73fb70259c5b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:22:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1065673572',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e505ef37-2986-40b0-a93d-6663f658f749,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0106885959194ffe9fdb6d6ced1c6e21',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:22:02Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:22:02Z,use_quota=True,user_id='2bf729e890a94334901003c39d13850e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:01.856469 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (16df12a5-00ff-4673-b9c8-d510a4d0a593) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:01.862816 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (16df12a5-00ff-4673-b9c8-d510a4d0a593) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:01.872951 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3b77065a-1b58-4a7a-8237-c5c2625f848c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:02.033340 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3b77065a-1b58-4a7a-8237-c5c2625f848c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'e505ef37-2986-40b0-a93d-6663f658f749', 'volume_name': 'volume-e505ef37-2986-40b0-a93d-6663f658f749', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:02.038580 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2d44f3d5-389e-4c4f-8431-b97c1f6dcfda) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:02.050772 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2d44f3d5-389e-4c4f-8431-b97c1f6dcfda) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:02.057180 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9b351ff2-7e04-4f11-a26f-157d955cf240) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:02.062083 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Volume e505ef37-2986-40b0-a93d-6663f658f749: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-e505ef37-2986-40b0-a93d-6663f658f749', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 21:22:02.063215 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Cloning e505ef37-2986-40b0-a93d-6663f658f749 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 21:22:02.064917 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:22:02.078092 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.013s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:22:02.092263 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 22, 2, 84173)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:22:02.093784 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:22:02.094481 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 21:22:02.111118 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.032s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:22:02.112249 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 21:22:02.123479 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 22, 2, 116127)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:22:02.123479 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:22:02.123479 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 21:22:02.138613 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:02.241528 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] creating snapshot='volume-e505ef37-2986-40b0-a93d-6663f658f749.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:22:02.707891 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-e505ef37-2986-40b0-a93d-6663f658f749.clone_snap' to 'volume-e505ef37-2986-40b0-a93d-6663f658f749' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:22:02.894175 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:22:02.895699 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:22:02.896263 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:22:02.916482 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:22:02.917397 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Marking volume e505ef37-2986-40b0-a93d-6663f658f749 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:22:02.931916 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to e505ef37-2986-40b0-a93d-6663f658f749. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 21:22:02.932701 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Creating volume glance metadata for volume e505ef37-2986-40b0-a93d-6663f658f749 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 21:22:02.945779 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=64,cinder:UPDATE=27,cinder:INSERT=7,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:03.024230 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9b351ff2-7e04-4f11-a26f-157d955cf240) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e505ef37-2986-40b0-a93d-6663f658f749', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:03.031151 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (87a692aa-2680-47b2-8374-588dd7240be8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:03.040710 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=57,cinder:UPDATE=24,cinder:DELETE=1,cinder:INSERT=18 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:03.050149 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Volume volume-e505ef37-2986-40b0-a93d-6663f658f749 (e505ef37-2986-40b0-a93d-6663f658f749): created successfully Sep 17 21:22:03.058068 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (87a692aa-2680-47b2-8374-588dd7240be8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:03.073721 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Flow 'volume_create_manager' (2c74cae1-911e-4596-9fff-d73bf7760bf9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:22:03.091738 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-19d1a129-b1b8-4cf9-9d2b-a4a8286a64b2 tempest-VolumesCloneTest-1022632628 None] Created volume successfully. Sep 17 21:22:04.700085 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:22:04.843473 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Flow 'volume_create_manager' (a994be2e-3014-4545-8727-97e400159ad0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:22:04.863605 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (948dbdf9-0a56-47de-824f-66cc2689becc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:04.900199 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (948dbdf9-0a56-47de-824f-66cc2689becc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:22:04Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-636520191',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2b28dfa5-311d-4a29-8e56-42f37ed7a8d6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0106885959194ffe9fdb6d6ced1c6e21',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:22:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=e505ef37-2986-40b0-a93d-6663f658f749,status='creating',terminated_at=None,updated_at=2022-09-17T21:22:05Z,use_quota=True,user_id='2bf729e890a94334901003c39d13850e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:04.910123 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (158713af-952a-4a78-a096-65e44440639c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:04.916525 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (158713af-952a-4a78-a096-65e44440639c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:04.925377 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (61beba62-f9d5-420c-abe4-df03fb57ca72) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:04.981565 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (61beba62-f9d5-420c-abe4-df03fb57ca72) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '2b28dfa5-311d-4a29-8e56-42f37ed7a8d6', 'volume_name': 'volume-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6', 'volume_size': 1, 'source_volid': 'e505ef37-2986-40b0-a93d-6663f658f749', 'source_volstatus': 'available'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:04.993974 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a525a9c8-d910-4b31-8925-dc2a7d29efbb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:04.999280 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a525a9c8-d910-4b31-8925-dc2a7d29efbb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:05.005738 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (71b68c99-2bf5-4797-a349-cbc55a57c735) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:05.008227 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Volume 2b28dfa5-311d-4a29-8e56-42f37ed7a8d6: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6', 'volume_size': 1, 'source_volid': 'e505ef37-2986-40b0-a93d-6663f658f749', 'source_volstatus': 'available'} Sep 17 21:22:05.025605 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:05.138017 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] creating snapshot='volume-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:22:05.383191 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] cloning 'volume-e505ef37-2986-40b0-a93d-6663f658f749@volume-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6.clone_snap' to 'volume-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:22:05.646385 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:22:05.647819 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:22:05.658048 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:22:05.723138 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:22:05.723929 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Marking volume 2b28dfa5-311d-4a29-8e56-42f37ed7a8d6 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:22:05.742649 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Copying metadata from source volume e505ef37-2986-40b0-a93d-6663f658f749 to 2b28dfa5-311d-4a29-8e56-42f37ed7a8d6. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:439}} Sep 17 21:22:05.790813 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (71b68c99-2bf5-4797-a349-cbc55a57c735) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6', 'volume_size': 1, 'source_volid': 'e505ef37-2986-40b0-a93d-6663f658f749', 'source_volstatus': 'available'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:05.797702 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fdb5f8eb-8b94-4bd9-8c6c-07d72f17e64f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:05.812792 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Volume volume-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6 (2b28dfa5-311d-4a29-8e56-42f37ed7a8d6): created successfully Sep 17 21:22:05.821563 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fdb5f8eb-8b94-4bd9-8c6c-07d72f17e64f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:05.830246 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Flow 'volume_create_manager' (a994be2e-3014-4545-8727-97e400159ad0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:22:05.857549 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-be2809f8-baf2-43ac-a147-c16973a551cd tempest-VolumesCloneTest-1022632628 None] Created volume successfully. Sep 17 21:22:07.663166 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:22:07.712979 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Flow 'volume_create_manager' (8e1c2b3d-bc48-456a-ae08-b333b2d44cbd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:22:07.732821 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (662bcc16-c13a-4c63-aa8a-32a22ed6a3a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:07.781417 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (662bcc16-c13a-4c63-aa8a-32a22ed6a3a8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:22:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-868220595',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=af52cbf0-aa0c-426f-b406-826658f94127,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0106885959194ffe9fdb6d6ced1c6e21',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:22:08Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:22:08Z,use_quota=True,user_id='2bf729e890a94334901003c39d13850e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:07.787220 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c6a5dfc2-d4b8-4636-8bd5-7b61cbd66fab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:07.792571 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c6a5dfc2-d4b8-4636-8bd5-7b61cbd66fab) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:07.797866 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8d401988-28fe-4eae-9d33-9cb0eb4a22d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:07.802893 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8d401988-28fe-4eae-9d33-9cb0eb4a22d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'af52cbf0-aa0c-426f-b406-826658f94127', 'volume_name': 'volume-af52cbf0-aa0c-426f-b406-826658f94127', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:07.808662 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5653a4ef-78a4-404a-8aaa-5a99289b0613) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:07.812914 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5653a4ef-78a4-404a-8aaa-5a99289b0613) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:07.819019 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (05b78cf3-a062-4460-af3b-995fb71e936d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:07.821086 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Volume af52cbf0-aa0c-426f-b406-826658f94127: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-af52cbf0-aa0c-426f-b406-826658f94127', 'volume_size': 1} Sep 17 21:22:07.821736 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] creating volume 'volume-af52cbf0-aa0c-426f-b406-826658f94127' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:22:07.823195 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:07.944231 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:22:07.944231 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:22:07.944231 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (05b78cf3-a062-4460-af3b-995fb71e936d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-af52cbf0-aa0c-426f-b406-826658f94127', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:07.944231 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dc086899-ac0c-4654-b4a8-5c85ea983798) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:07.968015 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Volume volume-af52cbf0-aa0c-426f-b406-826658f94127 (af52cbf0-aa0c-426f-b406-826658f94127): created successfully Sep 17 21:22:07.972205 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dc086899-ac0c-4654-b4a8-5c85ea983798) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:07.983091 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Flow 'volume_create_manager' (8e1c2b3d-bc48-456a-ae08-b333b2d44cbd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:22:08.008416 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-71142596-a7e6-4af7-a364-38b9015707b9 tempest-VolumesCloneTest-1022632628 None] Created volume successfully. Sep 17 21:22:09.516273 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:22:09.608495 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Flow 'volume_create_manager' (9155010d-956e-4b96-9297-dc23d27cb347) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:22:09.618044 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (748abb33-b03b-40b0-9fff-b66864118188) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:09.649366 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (748abb33-b03b-40b0-9fff-b66864118188) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:22:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1603215100',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=7003cd6c-fd3f-456b-8ff9-b42e1af95c83,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0106885959194ffe9fdb6d6ced1c6e21',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:22:09Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=af52cbf0-aa0c-426f-b406-826658f94127,status='creating',terminated_at=None,updated_at=2022-09-17T21:22:09Z,use_quota=True,user_id='2bf729e890a94334901003c39d13850e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:09.655577 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4196079d-8830-4225-97dc-c1b60bd1ad41) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:09.662082 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4196079d-8830-4225-97dc-c1b60bd1ad41) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:09.669419 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ca0ad1dd-57df-49a9-a7db-eee4514e2975) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:09.733792 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ca0ad1dd-57df-49a9-a7db-eee4514e2975) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '7003cd6c-fd3f-456b-8ff9-b42e1af95c83', 'volume_name': 'volume-7003cd6c-fd3f-456b-8ff9-b42e1af95c83', 'volume_size': 2, 'source_volid': 'af52cbf0-aa0c-426f-b406-826658f94127', 'source_volstatus': 'available'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:09.744937 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a4364c6-217d-4ab7-8411-76bab4abeaad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:09.766237 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a4364c6-217d-4ab7-8411-76bab4abeaad) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:09.778461 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a2d5968c-c14e-4e2d-a8e6-e6f8bf7536f5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:09.780087 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Volume 7003cd6c-fd3f-456b-8ff9-b42e1af95c83: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-7003cd6c-fd3f-456b-8ff9-b42e1af95c83', 'volume_size': 2, 'source_volid': 'af52cbf0-aa0c-426f-b406-826658f94127', 'source_volstatus': 'available'} Sep 17 21:22:09.818691 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:09.897756 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] creating snapshot='volume-7003cd6c-fd3f-456b-8ff9-b42e1af95c83.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:22:10.476816 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] cloning 'volume-af52cbf0-aa0c-426f-b406-826658f94127@volume-7003cd6c-fd3f-456b-8ff9-b42e1af95c83.clone_snap' to 'volume-7003cd6c-fd3f-456b-8ff9-b42e1af95c83' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:22:10.651886 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Volume volume-af52cbf0-aa0c-426f-b406-826658f94127 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:22:10.653014 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:22:10.653383 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:22:10.682389 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] resize volume 'volume-7003cd6c-fd3f-456b-8ff9-b42e1af95c83' from 1 to 2 {{(pid=133409) _extend_if_required /opt/stack/cinder/cinder/volume/drivers/rbd.py:812}} Sep 17 21:22:10.683316 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:10.878489 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:22:10.884882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a2d5968c-c14e-4e2d-a8e6-e6f8bf7536f5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7003cd6c-fd3f-456b-8ff9-b42e1af95c83', 'volume_size': 2, 'source_volid': 'af52cbf0-aa0c-426f-b406-826658f94127', 'source_volstatus': 'available'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:10.908351 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dbceb2a4-d4d0-4682-b86f-e10eb35b9180) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:22:10.926692 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Volume volume-7003cd6c-fd3f-456b-8ff9-b42e1af95c83 (7003cd6c-fd3f-456b-8ff9-b42e1af95c83): created successfully Sep 17 21:22:10.934660 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dbceb2a4-d4d0-4682-b86f-e10eb35b9180) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:22:10.950073 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Flow 'volume_create_manager' (9155010d-956e-4b96-9297-dc23d27cb347) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:22:10.982651 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c2d02697-1ada-4167-bf7d-b7aa2c342ba3 tempest-VolumesCloneTest-1022632628 None] Created volume successfully. Sep 17 21:22:11.863386 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] Acquiring lock "cinder-7003cd6c-fd3f-456b-8ff9-b42e1af95c83-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:22:11.876806 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] Lock "cinder-7003cd6c-fd3f-456b-8ff9-b42e1af95c83-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:22:11.918608 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:12.044693 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:22:12.060551 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] deleting rbd volume volume-7003cd6c-fd3f-456b-8ff9-b42e1af95c83 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:22:12.350960 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:22:12.386034 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] Volume volume-af52cbf0-aa0c-426f-b406-826658f94127 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:22:12.386804 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] deleting parent snapshot volume-7003cd6c-fd3f-456b-8ff9-b42e1af95c83.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:22:12.658408 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] Created reservations ['20dcd355-85ba-4c73-a13f-7b7281e97715', 'ab37b1f2-2c34-43da-a9c4-348a86229e8c', 'f452e62c-26a5-4ad4-8d17-2e67ed85f9d6', '8e5b5713-8d85-46df-b1f0-3da9cd47697f'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:22:12.763676 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:12.827233 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:12.895539 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:22:12.903697 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] Deleted volume successfully. Sep 17 21:22:12.928388 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b8905861-81ea-46b2-af0a-e0c15befba9d tempest-VolumesCloneTest-1022632628 None] Lock "cinder-7003cd6c-fd3f-456b-8ff9-b42e1af95c83-delete_volume" released by "delete_volume" :: held 1.051s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:22:13.145519 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] Acquiring lock "cinder-af52cbf0-aa0c-426f-b406-826658f94127-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:22:13.174338 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] Lock "cinder-af52cbf0-aa0c-426f-b406-826658f94127-delete_volume" acquired by "delete_volume" :: waited 0.032s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:22:13.291611 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:13.377418 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:22:13.377754 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] Volume volume-af52cbf0-aa0c-426f-b406-826658f94127 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:22:13.384318 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] deleting rbd volume volume-af52cbf0-aa0c-426f-b406-826658f94127 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:22:13.829566 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] Created reservations ['36fd6876-a590-4ac5-bc17-3ea003887925', '1bb78e2b-fe13-4dde-837c-7bcd1e224c5d', '506f3f61-f0a8-47b2-b3d3-974bfbef4fdc', '2dd60284-781c-43dd-a697-eac34c21d951'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:22:13.910681 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:13.986059 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:14.057191 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:22:14.061292 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] Deleted volume successfully. Sep 17 21:22:14.086572 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f114d238-0238-4d26-98ce-3b898f521e5b tempest-VolumesCloneTest-1022632628 None] Lock "cinder-af52cbf0-aa0c-426f-b406-826658f94127-delete_volume" released by "delete_volume" :: held 0.912s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:22:14.434668 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] Acquiring lock "cinder-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:22:14.452264 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] Lock "cinder-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:22:14.502976 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:14.654349 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:22:14.679704 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] deleting rbd volume volume-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:22:15.048321 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:22:15.102340 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] deleting parent snapshot volume-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:22:15.787038 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] Created reservations ['47715cac-91cb-4014-a955-911e07020947', '888700b8-2cd6-4c74-9937-4a4b6dc1c037', '4c781075-75fb-40c2-a799-2f5545991971', 'bf01843a-d78c-485b-88d3-b2bbb7fc9dd4'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:22:15.861373 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=55,cinder:UPDATE=28,cinder:DELETE=2,cinder:INSERT=15 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:15.863553 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=60,cinder:UPDATE=27,cinder:INSERT=9,cinder:DELETE=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:15.884155 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:15.940932 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:16.003733 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:22:16.015243 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] Deleted volume successfully. Sep 17 21:22:16.038934 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-f04f1052-a723-4b49-9a3c-0be5d926de88 tempest-VolumesCloneTest-1022632628 None] Lock "cinder-2b28dfa5-311d-4a29-8e56-42f37ed7a8d6-delete_volume" released by "delete_volume" :: held 1.587s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:22:16.787720 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] Acquiring lock "cinder-e505ef37-2986-40b0-a93d-6663f658f749-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:22:16.800220 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] Lock "cinder-e505ef37-2986-40b0-a93d-6663f658f749-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:22:16.860028 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:16.963054 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:22:16.973280 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] deleting rbd volume volume-e505ef37-2986-40b0-a93d-6663f658f749 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:22:17.221130 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:22:17.265000 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] deleting parent snapshot volume-e505ef37-2986-40b0-a93d-6663f658f749.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:22:17.755543 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] Created reservations ['38840909-bf3f-4ce9-9321-d85c8f283ee5', 'd3ed3c9c-8467-486a-8218-8f3e1f15a7a8', '814f1b65-64b6-496e-990c-39aab2f2f16e', 'bd8f8537-ee77-49fa-b0f7-70139e880160'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:22:17.812413 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:17.857425 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:17.905861 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:22:17.915661 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] Deleted volume successfully. Sep 17 21:22:17.943755 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-3d6af2d8-f914-4461-a5ce-03250425e1d5 tempest-VolumesCloneTest-1022632628 None] Lock "cinder-e505ef37-2986-40b0-a93d-6663f658f749-delete_volume" released by "delete_volume" :: held 1.143s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:22:26.825202 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-d2cde9ad-b9f5-49a9-b212-88c96842183f tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['ae1eae17-7f74-4fe2-86a8-cb5f4b308d74', '7be6485d-9bef-4847-a341-712e91f10aa9', 'baf986d7-267e-477b-ad4c-ef9b1c931b26', 'a615d0ea-3aec-4e68-a8ee-e98660cb6219'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:22:26.890730 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d2cde9ad-b9f5-49a9-b212-88c96842183f tempest-VolumesSnapshotTestJSON-2104722578 None] Delete snapshot completed successfully. Sep 17 21:22:26.896846 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d2cde9ad-b9f5-49a9-b212-88c96842183f tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-6e2b57cf-090b-4f00-9538-d73a42008839-delete_snapshot" released by "delete_snapshot" :: held 30.348s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:22:27.172205 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-75168bbc-53cb-4529-b57d-42953d2b29e0 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-dc22ca54-32ba-4883-8d34-e4aedd74fd5b-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:22:27.193868 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-75168bbc-53cb-4529-b57d-42953d2b29e0 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-dc22ca54-32ba-4883-8d34-e4aedd74fd5b-delete_snapshot" acquired by "delete_snapshot" :: waited 0.018s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:22:27.196066 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-75168bbc-53cb-4529-b57d-42953d2b29e0 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:22:43.402051 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=16,cinder:SELECT=22,cinder:DELETE=1,cinder:INSERT=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:53.413140 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=30,cinder:INSERT=6,cinder:UPDATE=20,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:22:57.479141 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-75168bbc-53cb-4529-b57d-42953d2b29e0 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['2851cb35-24f6-45c1-ba85-3087ad134e80', '449511dc-cb2c-4230-9e20-c5f418877f16', '2d76cc97-8621-4ab9-94fc-5a184baa5f17', '9469444f-093d-4d17-ac99-220a1cf3233d'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:22:57.534639 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-75168bbc-53cb-4529-b57d-42953d2b29e0 tempest-VolumesSnapshotTestJSON-2104722578 None] Delete snapshot completed successfully. Sep 17 21:22:57.544192 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-75168bbc-53cb-4529-b57d-42953d2b29e0 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-dc22ca54-32ba-4883-8d34-e4aedd74fd5b-delete_snapshot" released by "delete_snapshot" :: held 30.353s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:22:58.853286 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-95a1e785-a875-45d7-b5e0-325d9cb1aa4d req-9e3f4eeb-51f4-4f88-a530-306dbef8b11b tempest-VolumesSnapshotTestJSON-2104722578 None] Terminate volume connection completed successfully. Sep 17 21:22:58.853982 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-95a1e785-a875-45d7-b5e0-325d9cb1aa4d req-9e3f4eeb-51f4-4f88-a530-306dbef8b11b tempest-VolumesSnapshotTestJSON-2104722578 None] Deleting attachment aa7ef21b-4604-40f7-ad32-2060f4898ef1. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:23:01.030993 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:23:01.032270 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:01.135229 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:01.197793 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:23:05.279446 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:23:05.320422 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (83c9c34c-dca0-44f6-8028-9eb5852f8a70) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:23:05.326942 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (80ce0a68-d96b-4cd9-96b9-251a65a3c77d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:05.352819 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (80ce0a68-d96b-4cd9-96b9-251a65a3c77d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:23:05Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-514025449',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5e163a34-1613-41a5-82e1-4ba099dc9c28,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='335955c0be3743eda2541b7d04ed2e06',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:23:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:23:05Z,use_quota=True,user_id='fd15b9d354eb44b0bfa83bcdcf5c1384',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:05.358198 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0fbfc1c0-621e-431b-ba8e-efff2bca7587) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:05.370373 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0fbfc1c0-621e-431b-ba8e-efff2bca7587) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:05.377499 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d7d2f398-357a-4ceb-881d-b676898796a6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:05.387133 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d7d2f398-357a-4ceb-881d-b676898796a6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5e163a34-1613-41a5-82e1-4ba099dc9c28', 'volume_name': 'volume-5e163a34-1613-41a5-82e1-4ba099dc9c28', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:05.393330 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5fe52c43-3249-4518-8e8e-32a61570d9f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:05.398585 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5fe52c43-3249-4518-8e8e-32a61570d9f9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:05.406211 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f752a96e-cdaf-4550-92d8-fdcd1f033308) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:05.407676 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume 5e163a34-1613-41a5-82e1-4ba099dc9c28: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5e163a34-1613-41a5-82e1-4ba099dc9c28', 'volume_size': 1} Sep 17 21:23:05.408281 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] creating volume 'volume-5e163a34-1613-41a5-82e1-4ba099dc9c28' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:23:05.409062 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:05.538039 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:23:05.538884 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:23:05.544301 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f752a96e-cdaf-4550-92d8-fdcd1f033308) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5e163a34-1613-41a5-82e1-4ba099dc9c28', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:05.554107 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f71be412-ddf4-4207-acfe-3d16c3399a03) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:05.584630 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume volume-5e163a34-1613-41a5-82e1-4ba099dc9c28 (5e163a34-1613-41a5-82e1-4ba099dc9c28): created successfully Sep 17 21:23:05.589300 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f71be412-ddf4-4207-acfe-3d16c3399a03) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:05.595571 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (83c9c34c-dca0-44f6-8028-9eb5852f8a70) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:23:05.609496 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-58d74466-a95f-4ac6-ad6f-f372b52a0440 tempest-VolumesSnapshotTestJSON-2104722578 None] Created volume successfully. Sep 17 21:23:06.661354 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-06f8f195-4eff-40ba-94cc-2d71816596e1 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:07.682108 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-06f8f195-4eff-40ba-94cc-2d71816596e1 tempest-VolumesSnapshotTestJSON-2104722578 None] Create snapshot completed successfully Sep 17 21:23:09.365708 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:23:09.421739 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (6c6f17bc-9a51-4829-9502-0a33817bb447) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:23:09.434201 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (30a72513-a04f-4471-baf3-11cacd0233f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:09.463314 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (30a72513-a04f-4471-baf3-11cacd0233f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:23:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-45011409',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3ad55a54-61ff-4d48-8970-4a4950210c1c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='335955c0be3743eda2541b7d04ed2e06',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:23:09Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=b76a8388-f48e-4bc7-b8c2-5eaabbe11a13,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:23:09Z,use_quota=True,user_id='fd15b9d354eb44b0bfa83bcdcf5c1384',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:09.476721 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f923c141-e526-49e3-9757-c0b19715398c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:09.487491 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f923c141-e526-49e3-9757-c0b19715398c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:09.495874 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4823a59c-83f3-498a-b6a0-9f64291d616d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:09.501893 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4823a59c-83f3-498a-b6a0-9f64291d616d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '3ad55a54-61ff-4d48-8970-4a4950210c1c', 'volume_name': 'volume-3ad55a54-61ff-4d48-8970-4a4950210c1c', 'volume_size': 2, 'snapshot_id': 'b76a8388-f48e-4bc7-b8c2-5eaabbe11a13'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:09.516482 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9747b32d-fe60-4eba-ae5c-029384371af6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:09.523112 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9747b32d-fe60-4eba-ae5c-029384371af6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:09.532889 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8987d3c-6454-48c7-a96f-6fc1c5a0d632) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:09.534564 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume 3ad55a54-61ff-4d48-8970-4a4950210c1c: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-3ad55a54-61ff-4d48-8970-4a4950210c1c', 'volume_size': 2, 'snapshot_id': 'b76a8388-f48e-4bc7-b8c2-5eaabbe11a13'} Sep 17 21:23:09.562520 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] cloning volumes/volume-5e163a34-1613-41a5-82e1-4ba099dc9c28@snapshot-b76a8388-f48e-4bc7-b8c2-5eaabbe11a13 to volume-3ad55a54-61ff-4d48-8970-4a4950210c1c {{(pid=133409) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 17 21:23:09.563230 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:09.615260 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:09.792521 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:23:09.793008 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:23:09.801315 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:09.953789 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8987d3c-6454-48c7-a96f-6fc1c5a0d632) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3ad55a54-61ff-4d48-8970-4a4950210c1c', 'volume_size': 2, 'snapshot_id': 'b76a8388-f48e-4bc7-b8c2-5eaabbe11a13'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:09.958816 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9bfdb255-557f-47ed-89fd-2432732df067) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:09.971707 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume volume-3ad55a54-61ff-4d48-8970-4a4950210c1c (3ad55a54-61ff-4d48-8970-4a4950210c1c): created successfully Sep 17 21:23:09.975855 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9bfdb255-557f-47ed-89fd-2432732df067) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:09.981881 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (6c6f17bc-9a51-4829-9502-0a33817bb447) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:23:10.003611 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6891d743-d8ce-456d-aa7a-9b09cd20ea07 tempest-VolumesSnapshotTestJSON-2104722578 None] Created volume successfully. Sep 17 21:23:10.464686 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d286e59b-7ee5-4cdc-ae88-dd754c29c1f5 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-3ad55a54-61ff-4d48-8970-4a4950210c1c-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:23:10.476026 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d286e59b-7ee5-4cdc-ae88-dd754c29c1f5 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-3ad55a54-61ff-4d48-8970-4a4950210c1c-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:23:10.548599 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d286e59b-7ee5-4cdc-ae88-dd754c29c1f5 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:10.656030 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d286e59b-7ee5-4cdc-ae88-dd754c29c1f5 tempest-VolumesSnapshotTestJSON-2104722578 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:23:10.664149 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d286e59b-7ee5-4cdc-ae88-dd754c29c1f5 tempest-VolumesSnapshotTestJSON-2104722578 None] deleting rbd volume volume-3ad55a54-61ff-4d48-8970-4a4950210c1c {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:23:11.078239 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-d286e59b-7ee5-4cdc-ae88-dd754c29c1f5 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['6a53fda1-1a4f-439a-ab03-8dfbdc59c534', '1bc90c6f-f749-4f23-9f0f-76e67ccc2a20', '4a46e90e-190a-4651-b564-d705481cd63b', 'caf9e16f-6be2-4e49-927a-abf724a09051'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:23:11.135464 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d286e59b-7ee5-4cdc-ae88-dd754c29c1f5 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:11.207120 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d286e59b-7ee5-4cdc-ae88-dd754c29c1f5 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:11.251541 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-d286e59b-7ee5-4cdc-ae88-dd754c29c1f5 tempest-VolumesSnapshotTestJSON-2104722578 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:23:11.258200 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d286e59b-7ee5-4cdc-ae88-dd754c29c1f5 tempest-VolumesSnapshotTestJSON-2104722578 None] Deleted volume successfully. Sep 17 21:23:11.289261 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d286e59b-7ee5-4cdc-ae88-dd754c29c1f5 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-3ad55a54-61ff-4d48-8970-4a4950210c1c-delete_volume" released by "delete_volume" :: held 0.813s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:23:12.005515 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:23:12.030943 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (4298e373-fa01-4c54-872f-1e81a74b397f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:23:12.037946 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e7cc36c1-b688-4de1-965d-2b18315d0c18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:12.054955 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e7cc36c1-b688-4de1-965d-2b18315d0c18) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:23:12Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1698296534',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=aed7642a-f877-4315-a849-2a508107c10b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='335955c0be3743eda2541b7d04ed2e06',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:23:12Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:23:12Z,use_quota=True,user_id='fd15b9d354eb44b0bfa83bcdcf5c1384',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:12.060044 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6d77c425-7e24-4547-a1dd-82547b0e5205) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:12.063989 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6d77c425-7e24-4547-a1dd-82547b0e5205) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:12.068967 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1958b8cb-cd9f-4b3a-9fc5-9c8a41322d45) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:12.073210 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1958b8cb-cd9f-4b3a-9fc5-9c8a41322d45) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'aed7642a-f877-4315-a849-2a508107c10b', 'volume_name': 'volume-aed7642a-f877-4315-a849-2a508107c10b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:12.077941 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d7b0d915-d852-4930-a723-957ddbb3ec19) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:12.082456 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d7b0d915-d852-4930-a723-957ddbb3ec19) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:12.087668 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6e81ce49-2c62-484d-b724-c333b128a309) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:12.089098 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Volume aed7642a-f877-4315-a849-2a508107c10b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-aed7642a-f877-4315-a849-2a508107c10b', 'volume_size': 1} Sep 17 21:23:12.089574 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] creating volume 'volume-aed7642a-f877-4315-a849-2a508107c10b' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:23:12.090334 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:12.193644 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:23:12.194405 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:23:12.200259 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6e81ce49-2c62-484d-b724-c333b128a309) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-aed7642a-f877-4315-a849-2a508107c10b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:12.212769 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (188ff169-f650-49f4-a011-5c255fa60013) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:12.231164 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Volume volume-aed7642a-f877-4315-a849-2a508107c10b (aed7642a-f877-4315-a849-2a508107c10b): created successfully Sep 17 21:23:12.236640 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (188ff169-f650-49f4-a011-5c255fa60013) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:12.245941 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (4298e373-fa01-4c54-872f-1e81a74b397f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:23:12.261005 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b4044ad2-24eb-4aff-a561-1fce52a9cb7f tempest-VolumesSnapshotTestJSON-2104722578 None] Created volume successfully. Sep 17 21:23:13.446155 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-eb6257a7-3ee7-4bda-b9ea-700427fd6ffe tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:13.507965 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=53,cinder:UPDATE=22,cinder:INSERT=4,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:13.817279 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-eb6257a7-3ee7-4bda-b9ea-700427fd6ffe tempest-VolumesSnapshotTestJSON-2104722578 None] Create snapshot completed successfully Sep 17 21:23:15.325042 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:23:15.365387 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (ab85abb7-c961-4362-8fd7-3492aabc6ef3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:23:15.375016 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d1c73224-e132-4fde-8ebe-b2ed1013b757) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:15.397701 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d1c73224-e132-4fde-8ebe-b2ed1013b757) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:23:15Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1290271451',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b66881b2-150d-4e42-957d-2bd2f79012cf,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='335955c0be3743eda2541b7d04ed2e06',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:23:15Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=4b2f7111-55af-477f-9312-3f101c790af9,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:23:15Z,use_quota=True,user_id='fd15b9d354eb44b0bfa83bcdcf5c1384',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:15.406902 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8f69c910-49b9-4761-a62e-1acf317634af) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:15.411985 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8f69c910-49b9-4761-a62e-1acf317634af) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:15.418194 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f22ae563-ae3a-4d21-bf53-b1a0b197fe39) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:15.422880 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f22ae563-ae3a-4d21-bf53-b1a0b197fe39) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'b66881b2-150d-4e42-957d-2bd2f79012cf', 'volume_name': 'volume-b66881b2-150d-4e42-957d-2bd2f79012cf', 'volume_size': 1, 'snapshot_id': '4b2f7111-55af-477f-9312-3f101c790af9'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:15.427987 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4cd004df-ec7a-4101-9776-3bfda12c2361) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:15.432131 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4cd004df-ec7a-4101-9776-3bfda12c2361) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:15.437452 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (55dcda98-f047-4d34-b50a-e2b877fa50fa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:15.438623 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume b66881b2-150d-4e42-957d-2bd2f79012cf: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-b66881b2-150d-4e42-957d-2bd2f79012cf', 'volume_size': 1, 'snapshot_id': '4b2f7111-55af-477f-9312-3f101c790af9'} Sep 17 21:23:15.461371 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] cloning volumes/volume-aed7642a-f877-4315-a849-2a508107c10b@snapshot-4b2f7111-55af-477f-9312-3f101c790af9 to volume-b66881b2-150d-4e42-957d-2bd2f79012cf {{(pid=133409) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 17 21:23:15.462292 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:15.532387 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:15.730904 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:23:15.731490 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:23:15.765678 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (55dcda98-f047-4d34-b50a-e2b877fa50fa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b66881b2-150d-4e42-957d-2bd2f79012cf', 'volume_size': 1, 'snapshot_id': '4b2f7111-55af-477f-9312-3f101c790af9'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:15.783671 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (db9d52f5-b834-4449-855b-08015e0bbb94) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:23:15.803883 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume volume-b66881b2-150d-4e42-957d-2bd2f79012cf (b66881b2-150d-4e42-957d-2bd2f79012cf): created successfully Sep 17 21:23:15.807536 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (db9d52f5-b834-4449-855b-08015e0bbb94) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:23:15.813312 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Flow 'volume_create_manager' (ab85abb7-c961-4362-8fd7-3492aabc6ef3) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:23:15.849039 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8eb24fcf-382a-4a67-a792-6a6e5c909863 tempest-VolumesSnapshotTestJSON-2104722578 None] Created volume successfully. Sep 17 21:23:16.455530 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ed423d51-2e5c-463e-8322-762231ad1d25 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-b66881b2-150d-4e42-957d-2bd2f79012cf-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:23:16.465220 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ed423d51-2e5c-463e-8322-762231ad1d25 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-b66881b2-150d-4e42-957d-2bd2f79012cf-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:23:16.534012 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ed423d51-2e5c-463e-8322-762231ad1d25 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:16.698498 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ed423d51-2e5c-463e-8322-762231ad1d25 tempest-VolumesSnapshotTestJSON-2104722578 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:23:16.714780 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ed423d51-2e5c-463e-8322-762231ad1d25 tempest-VolumesSnapshotTestJSON-2104722578 None] deleting rbd volume volume-b66881b2-150d-4e42-957d-2bd2f79012cf {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:23:17.155713 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-ed423d51-2e5c-463e-8322-762231ad1d25 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['ec7d36c9-b742-48ff-8b96-7860adf6f156', '353467f7-5ac9-43a0-8572-67076469ba23', '3fbcbd58-df13-45f4-af39-c48bd38c8038', '616179d2-3e28-4fb7-8364-f7b58765dd2c'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:23:17.173860 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=66,cinder:INSERT=8,cinder:UPDATE=21,cinder:DELETE=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:17.230733 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ed423d51-2e5c-463e-8322-762231ad1d25 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:17.273714 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ed423d51-2e5c-463e-8322-762231ad1d25 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:17.361551 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-ed423d51-2e5c-463e-8322-762231ad1d25 tempest-VolumesSnapshotTestJSON-2104722578 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:23:17.385514 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ed423d51-2e5c-463e-8322-762231ad1d25 tempest-VolumesSnapshotTestJSON-2104722578 None] Deleted volume successfully. Sep 17 21:23:17.410714 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ed423d51-2e5c-463e-8322-762231ad1d25 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-b66881b2-150d-4e42-957d-2bd2f79012cf-delete_volume" released by "delete_volume" :: held 0.945s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:23:17.773665 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-af0fc977-3eba-429a-8ed6-deeb200b5941 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-4b2f7111-55af-477f-9312-3f101c790af9-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:23:17.794932 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-af0fc977-3eba-429a-8ed6-deeb200b5941 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-4b2f7111-55af-477f-9312-3f101c790af9-delete_snapshot" acquired by "delete_snapshot" :: waited 0.022s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:23:17.796733 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-af0fc977-3eba-429a-8ed6-deeb200b5941 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:18.653772 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-af0fc977-3eba-429a-8ed6-deeb200b5941 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['a5081d24-4d0e-40fa-983a-a5af0774b146', '013485f0-339b-4146-9e66-02d0b8b01b24', 'f311badf-b22d-4894-a9dd-61a1f654d474', 'f1e83815-591e-4d43-96e3-88299e0d92a0'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:23:18.711441 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-af0fc977-3eba-429a-8ed6-deeb200b5941 tempest-VolumesSnapshotTestJSON-2104722578 None] Delete snapshot completed successfully. Sep 17 21:23:18.718364 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-af0fc977-3eba-429a-8ed6-deeb200b5941 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-4b2f7111-55af-477f-9312-3f101c790af9-delete_snapshot" released by "delete_snapshot" :: held 0.923s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:23:18.914955 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-aed7642a-f877-4315-a849-2a508107c10b-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:23:18.930679 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-aed7642a-f877-4315-a849-2a508107c10b-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:23:18.980851 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:19.049881 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:23:19.049881 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume volume-aed7642a-f877-4315-a849-2a508107c10b is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:23:19.054779 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] deleting rbd volume volume-aed7642a-f877-4315-a849-2a508107c10b {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:23:19.384060 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['16c2913a-ad17-4a94-b735-93a5ac38629b', 'fdcdff9c-85dc-49a7-9770-90f07f604c7d', '8193f4b7-481c-4f95-9f36-9da1af58ff76', '319fceab-6fa0-4e4b-a970-d202bdcbec41'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:23:19.468714 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:19.553175 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:19.607731 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:23:19.613729 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] Deleted volume successfully. Sep 17 21:23:19.632083 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-00ca6467-bc6e-456e-8287-ccc88ffc1667 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-aed7642a-f877-4315-a849-2a508107c10b-delete_volume" released by "delete_volume" :: held 0.701s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:23:20.133131 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-87d57aec-f7dc-489e-8dd3-bec18774d334 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-b76a8388-f48e-4bc7-b8c2-5eaabbe11a13-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:23:20.147910 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-87d57aec-f7dc-489e-8dd3-bec18774d334 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-b76a8388-f48e-4bc7-b8c2-5eaabbe11a13-delete_snapshot" acquired by "delete_snapshot" :: waited 0.014s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:23:20.151646 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-87d57aec-f7dc-489e-8dd3-bec18774d334 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:20.708689 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-87d57aec-f7dc-489e-8dd3-bec18774d334 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['3b2ddee1-c8db-4159-ab62-0c8248de3d9b', '7673d2b5-7878-4d64-9efc-001e45dd3ff4', '6e79e571-c203-45c3-8deb-52679d69ec4f', 'e87af4ad-4338-432b-a767-c9fcbb45eb75'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:23:20.778430 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-87d57aec-f7dc-489e-8dd3-bec18774d334 tempest-VolumesSnapshotTestJSON-2104722578 None] Delete snapshot completed successfully. Sep 17 21:23:20.785916 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-87d57aec-f7dc-489e-8dd3-bec18774d334 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-b76a8388-f48e-4bc7-b8c2-5eaabbe11a13-delete_snapshot" released by "delete_snapshot" :: held 0.639s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:23:21.259053 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-5e163a34-1613-41a5-82e1-4ba099dc9c28-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:23:21.275560 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-5e163a34-1613-41a5-82e1-4ba099dc9c28-delete_volume" acquired by "delete_volume" :: waited 0.020s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:23:21.310858 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=61,cinder:UPDATE=36,cinder:INSERT=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:21.314357 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:21.384722 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:23:21.385163 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume volume-5e163a34-1613-41a5-82e1-4ba099dc9c28 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:23:21.392557 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] deleting rbd volume volume-5e163a34-1613-41a5-82e1-4ba099dc9c28 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:23:21.656729 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['8d168b08-efa6-495f-abaf-dc0f6cfc9929', '4d59b109-5f93-4cb5-8a0d-5ec728f2239a', 'a882d82a-09de-413b-b3d3-b074b344c7d1', '76286dfc-c413-48be-b75d-151ce6dfbcf1'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:23:21.735535 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:21.781492 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:21.813877 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:23:21.818819 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] Deleted volume successfully. Sep 17 21:23:21.837478 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2680fc21-6da4-44c1-99d3-26131d29e658 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-5e163a34-1613-41a5-82e1-4ba099dc9c28-delete_volume" released by "delete_volume" :: held 0.562s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:23:22.648365 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-f21f3370-6eb4-457e-9061-0f3b2e826e13-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:23:22.662662 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-f21f3370-6eb4-457e-9061-0f3b2e826e13-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:23:22.771873 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:22.879170 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:23:22.889944 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] deleting rbd volume volume-f21f3370-6eb4-457e-9061-0f3b2e826e13 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:23:23.163577 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:23:23.213898 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] deleting parent snapshot volume-f21f3370-6eb4-457e-9061-0f3b2e826e13.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:23:23.680970 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['f3a6da2f-892c-4b1e-8616-c187651cbcaa', '7c530a0b-b4a6-4dcb-b3d4-a05bd94d6645', 'd85cba2e-9228-409a-bac4-43fb93345a96', '7163f0e1-7fc1-4e08-a169-3d65451489a4'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:23:23.738336 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:23.811179 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:23.894551 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:23:23.899114 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] Deleted volume successfully. Sep 17 21:23:23.919249 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6c67094f-1c5d-4f54-a0fc-7062aa31b007 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-f21f3370-6eb4-457e-9061-0f3b2e826e13-delete_volume" released by "delete_volume" :: held 1.257s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:23:23.931460 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-624f17b7-a6ee-4574-86a6-4a58086c1abd tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-f6a42616-014d-4a50-80ab-633d0e60c8cb-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:23:23.950348 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-624f17b7-a6ee-4574-86a6-4a58086c1abd tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-f6a42616-014d-4a50-80ab-633d0e60c8cb-delete_snapshot" acquired by "delete_snapshot" :: waited 0.019s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:23:23.952927 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-624f17b7-a6ee-4574-86a6-4a58086c1abd tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:24.778503 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=33,cinder:SELECT=50,cinder:DELETE=4,cinder:INSERT=13 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:24.811838 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-624f17b7-a6ee-4574-86a6-4a58086c1abd tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['466c53b1-36c5-43db-a820-ee3fb9515614', '60f13492-aa75-4962-b0fe-28d991cd78e2', '91bad582-3ce0-473a-bf81-b0f18d55ae13', 'b608664f-7d09-4141-a836-d5b283755b50'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:23:24.909032 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-624f17b7-a6ee-4574-86a6-4a58086c1abd tempest-VolumesSnapshotTestJSON-2104722578 None] Delete snapshot completed successfully. Sep 17 21:23:24.918501 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-624f17b7-a6ee-4574-86a6-4a58086c1abd tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-f6a42616-014d-4a50-80ab-633d0e60c8cb-delete_snapshot" released by "delete_snapshot" :: held 0.968s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:23:25.072671 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] Acquiring lock "cinder-5b712483-f63f-4381-9968-4e3fa55f72b6-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:23:25.085159 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-5b712483-f63f-4381-9968-4e3fa55f72b6-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:23:25.156114 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:25.219831 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:23:25.220346 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] Volume volume-5b712483-f63f-4381-9968-4e3fa55f72b6 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:23:25.227691 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] deleting rbd volume volume-5b712483-f63f-4381-9968-4e3fa55f72b6 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:23:25.527019 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] Created reservations ['60eba2e9-8502-4f0d-bfc6-8cc0a11bd820', 'a6b5242b-2e8e-4879-bdac-e3686ee3b687', '5491f835-5915-4f40-a2cd-c06f3bed71e6', '58897221-6824-4634-9892-142c64af3bef'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:23:25.600293 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:25.667157 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:23:25.722730 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:23:25.728191 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] Deleted volume successfully. Sep 17 21:23:25.743980 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-14922531-5eca-4d96-ae17-642416fd6550 tempest-VolumesSnapshotTestJSON-2104722578 None] Lock "cinder-5b712483-f63f-4381-9968-4e3fa55f72b6-delete_volume" released by "delete_volume" :: held 0.659s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:23:43.669061 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=21,cinder:UPDATE=19,cinder:INSERT=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:23:43.670162 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=46,cinder:INSERT=8,cinder:UPDATE=30,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:01.032926 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:24:01.035103 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:01.115083 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:01.175888 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:24:03.653108 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:10.313665 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:10.344161 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (a0b0c5e4-aeb3-4f56-b3d0-1d6d319b71e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:10.352200 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (27d49425-ad69-4762-9575-0acc7123742c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:10.370260 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (27d49425-ad69-4762-9575-0acc7123742c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1912491646',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4f5efa55-03ee-4386-ad16-0c42d83a2e3b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b99d2a0b4d7e40a38e682fc147e968af',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:10Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:10Z,use_quota=True,user_id='342e60103c9b45bcb348b6f8bcd686ca',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:10.375553 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b4942e38-1702-4a57-b57e-078ad24a8d6d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:10.379798 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b4942e38-1702-4a57-b57e-078ad24a8d6d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:10.385581 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a492898b-3719-4781-9163-b6fad710b3c6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:10.402695 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e9f28453-1128-4437-b7b9-180f8121b4c1 tempest-GroupSnapshotsV319Test-1114553254 None] Group tempest-GroupSnapshotsV319Test-Group-537873322: creating Sep 17 21:24:10.437678 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e9f28453-1128-4437-b7b9-180f8121b4c1 tempest-GroupSnapshotsV319Test-1114553254 None] Group tempest-GroupSnapshotsV319Test-Group-537873322: created successfully Sep 17 21:24:10.448426 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e9f28453-1128-4437-b7b9-180f8121b4c1 tempest-GroupSnapshotsV319Test-1114553254 None] Create group completed successfully. Sep 17 21:24:10.488823 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a492898b-3719-4781-9163-b6fad710b3c6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '4f5efa55-03ee-4386-ad16-0c42d83a2e3b', 'volume_name': 'volume-4f5efa55-03ee-4386-ad16-0c42d83a2e3b', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:10.505885 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (31e5b529-ce34-4bea-b55c-b3236167795d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:10.510258 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (31e5b529-ce34-4bea-b55c-b3236167795d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:10.516259 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (00e5c634-c81d-46d8-a1b8-b80b644eea40) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:10.517550 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Volume 4f5efa55-03ee-4386-ad16-0c42d83a2e3b: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-4f5efa55-03ee-4386-ad16-0c42d83a2e3b', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 21:24:10.518122 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Cloning 4f5efa55-03ee-4386-ad16-0c42d83a2e3b from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 21:24:10.518858 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:24:10.528000 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:24:10.539527 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 24, 10, 533265)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:24:10.540222 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:24:10.540883 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 21:24:10.546615 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.019s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:24:10.548699 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 21:24:10.558683 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 24, 10, 554216)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:24:10.558798 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:24:10.559330 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 21:24:10.576723 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:10.631152 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] creating snapshot='volume-4f5efa55-03ee-4386-ad16-0c42d83a2e3b.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:24:11.583909 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-4f5efa55-03ee-4386-ad16-0c42d83a2e3b.clone_snap' to 'volume-4f5efa55-03ee-4386-ad16-0c42d83a2e3b' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:24:11.761502 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:24:11.763248 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:11.763723 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:11.800453 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:24:11.801542 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Marking volume 4f5efa55-03ee-4386-ad16-0c42d83a2e3b as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:24:11.813022 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to 4f5efa55-03ee-4386-ad16-0c42d83a2e3b. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 21:24:11.813442 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Creating volume glance metadata for volume 4f5efa55-03ee-4386-ad16-0c42d83a2e3b backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 21:24:11.899069 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (00e5c634-c81d-46d8-a1b8-b80b644eea40) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4f5efa55-03ee-4386-ad16-0c42d83a2e3b', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:11.904971 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8580ce71-1aab-40df-8418-37235ac4365e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:11.919905 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Volume volume-4f5efa55-03ee-4386-ad16-0c42d83a2e3b (4f5efa55-03ee-4386-ad16-0c42d83a2e3b): created successfully Sep 17 21:24:11.946627 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8580ce71-1aab-40df-8418-37235ac4365e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:11.957285 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (a0b0c5e4-aeb3-4f56-b3d0-1d6d319b71e1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:11.982755 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-8fcbc585-cee8-494a-ad78-e9604ad99c7f tempest-TestVolumeBootPattern-747170341 None] Created volume successfully. Sep 17 21:24:11.993336 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:12.017399 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Flow 'volume_create_manager' (01bccc19-0267-4bf6-af3c-7cb773458cf9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:12.035120 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (115e1d24-1f58-4457-a9d4-d375273a2c29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:12.060285 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (115e1d24-1f58-4457-a9d4-d375273a2c29) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:12Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsV319Test-Volume-519463869',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=7185a159-1846-49b1-88f9-71b7682f81f5,host='ncdv-master-1@ceph#ceph',id=7eb128f9-37ca-4a8e-8995-dfdc371914eb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='93c7626dedfb4acbaec8d2627c570b80',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:12Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:12Z,use_quota=True,user_id='57eafbe4f20f4c40b007fb5478205a9f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(b3cd01af-47bc-4be7-82f8-e9cdb83b4407),volume_type_id=b3cd01af-47bc-4be7-82f8-e9cdb83b4407)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:12.066868 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e5d9a664-6422-4edd-bb0a-b887bf53062c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:12.072040 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e5d9a664-6422-4edd-bb0a-b887bf53062c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:12.079346 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bb51cdd7-dcfc-4f8c-bc4f-8edd2309a345) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:12.084822 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bb51cdd7-dcfc-4f8c-bc4f-8edd2309a345) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '7eb128f9-37ca-4a8e-8995-dfdc371914eb', 'volume_name': 'volume-7eb128f9-37ca-4a8e-8995-dfdc371914eb', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:12.091977 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (82faf935-174d-4fff-aec7-a480c0a062d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:12.102957 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (82faf935-174d-4fff-aec7-a480c0a062d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:12.112305 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0b3613ff-fe1e-4230-833a-35cc0cdc57cb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:12.113388 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Volume 7eb128f9-37ca-4a8e-8995-dfdc371914eb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-7eb128f9-37ca-4a8e-8995-dfdc371914eb', 'volume_size': 1} Sep 17 21:24:12.113807 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] creating volume 'volume-7eb128f9-37ca-4a8e-8995-dfdc371914eb' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:24:12.114587 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:12.191967 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:12.194573 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:12.198892 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0b3613ff-fe1e-4230-833a-35cc0cdc57cb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7eb128f9-37ca-4a8e-8995-dfdc371914eb', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:12.203260 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (59c54785-8441-4bb8-bdb0-a528e5400169) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:12.219485 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Volume volume-7eb128f9-37ca-4a8e-8995-dfdc371914eb (7eb128f9-37ca-4a8e-8995-dfdc371914eb): created successfully Sep 17 21:24:12.226972 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (59c54785-8441-4bb8-bdb0-a528e5400169) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:12.239282 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Flow 'volume_create_manager' (01bccc19-0267-4bf6-af3c-7cb773458cf9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:12.251101 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7f2d9c81-5476-4e57-89c1-f2ac6d1aa92a tempest-GroupSnapshotsV319Test-1114553254 None] Created volume successfully. Sep 17 21:24:12.762953 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-00162d32-5fc5-4631-8175-693c4553d323 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:13.358026 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-82e30da5-045d-4d2a-90eb-b96064a5c0a8 tempest-GroupSnapshotsV319Test-1114553254 None] GroupSnapshot a49e30b0-b2bd-49d5-ad2c-cb46ebe99cc9: creating. Sep 17 21:24:13.400278 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-82e30da5-045d-4d2a-90eb-b96064a5c0a8 tempest-GroupSnapshotsV319Test-1114553254 None] Group snapshot a49e30b0-b2bd-49d5-ad2c-cb46ebe99cc9: creating. {{(pid=133409) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4121}} Sep 17 21:24:13.425537 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-82e30da5-045d-4d2a-90eb-b96064a5c0a8 tempest-GroupSnapshotsV319Test-1114553254 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:13.738070 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=44,cinder:UPDATE=12,cinder:INSERT=11,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:13.750926 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-00162d32-5fc5-4631-8175-693c4553d323 tempest-TestVolumeBootPattern-747170341 None] Create snapshot completed successfully Sep 17 21:24:13.767254 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-82e30da5-045d-4d2a-90eb-b96064a5c0a8 tempest-GroupSnapshotsV319Test-1114553254 None] group_snapshot a49e30b0-b2bd-49d5-ad2c-cb46ebe99cc9: created successfully Sep 17 21:24:15.002222 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3522410d-7bdb-447d-b6b8-91fcab5c4ae4 tempest-GroupSnapshotsV319Test-1114553254 None] group_snapshot a49e30b0-b2bd-49d5-ad2c-cb46ebe99cc9: deleting Sep 17 21:24:15.027748 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-3522410d-7bdb-447d-b6b8-91fcab5c4ae4 tempest-GroupSnapshotsV319Test-1114553254 None] group_snapshot a49e30b0-b2bd-49d5-ad2c-cb46ebe99cc9: deleting {{(pid=133409) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4296}} Sep 17 21:24:15.050268 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3522410d-7bdb-447d-b6b8-91fcab5c4ae4 tempest-GroupSnapshotsV319Test-1114553254 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:15.740555 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-3522410d-7bdb-447d-b6b8-91fcab5c4ae4 tempest-GroupSnapshotsV319Test-1114553254 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsV319Test-volume-type-527052427 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsV319Test-volume-type-527052427, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:24:15.742394 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-3522410d-7bdb-447d-b6b8-91fcab5c4ae4 tempest-GroupSnapshotsV319Test-1114553254 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsV319Test-volume-type-527052427 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsV319Test-volume-type-527052427, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:24:15.772759 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-3522410d-7bdb-447d-b6b8-91fcab5c4ae4 tempest-GroupSnapshotsV319Test-1114553254 None] Created reservations ['901fca82-cec8-48fa-adb4-93b3d069b283', '348dd2b8-b49d-4525-882a-21f7d9965b0f', '3e03fa63-170a-4493-8f2e-6257ba47ade5', '1da4baed-c155-4ea3-8aa3-dcce58b25db0'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:15.900703 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3522410d-7bdb-447d-b6b8-91fcab5c4ae4 tempest-GroupSnapshotsV319Test-1114553254 None] group_snapshot a49e30b0-b2bd-49d5-ad2c-cb46ebe99cc9: deleted successfully Sep 17 21:24:16.575833 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db657336-3c12-4d47-b2d9-7b2e4d3ec67f tempest-GroupSnapshotsV319Test-1114553254 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:16.636947 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db657336-3c12-4d47-b2d9-7b2e4d3ec67f tempest-GroupSnapshotsV319Test-1114553254 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:24:16.637372 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db657336-3c12-4d47-b2d9-7b2e4d3ec67f tempest-GroupSnapshotsV319Test-1114553254 None] Volume volume-7eb128f9-37ca-4a8e-8995-dfdc371914eb is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:24:16.642011 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db657336-3c12-4d47-b2d9-7b2e4d3ec67f tempest-GroupSnapshotsV319Test-1114553254 None] deleting rbd volume volume-7eb128f9-37ca-4a8e-8995-dfdc371914eb {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:24:16.838764 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=68,cinder:UPDATE=13,cinder:INSERT=17,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:16.864717 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-db657336-3c12-4d47-b2d9-7b2e4d3ec67f tempest-GroupSnapshotsV319Test-1114553254 None] Created reservations ['83461570-2d55-4d7a-a5e7-d87e56986b53'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:16.885144 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-db657336-3c12-4d47-b2d9-7b2e4d3ec67f tempest-GroupSnapshotsV319Test-1114553254 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsV319Test-volume-type-527052427 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsV319Test-volume-type-527052427, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:24:16.901503 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-db657336-3c12-4d47-b2d9-7b2e4d3ec67f tempest-GroupSnapshotsV319Test-1114553254 None] Created reservations ['8f23371e-9e2e-4127-a5c0-ead56536e918', 'd4f9e5f8-6b05-44e5-8db2-d4240a35eb7b', '089f517c-3c6b-4c9d-93d6-9fd134512355', 'c71de6f3-9a5f-48e8-ae65-4e5345b319c3'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:16.988848 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db657336-3c12-4d47-b2d9-7b2e4d3ec67f tempest-GroupSnapshotsV319Test-1114553254 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:17.018557 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-db657336-3c12-4d47-b2d9-7b2e4d3ec67f tempest-GroupSnapshotsV319Test-1114553254 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:17.056291 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-db657336-3c12-4d47-b2d9-7b2e4d3ec67f tempest-GroupSnapshotsV319Test-1114553254 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:24:17.067643 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-db657336-3c12-4d47-b2d9-7b2e4d3ec67f tempest-GroupSnapshotsV319Test-1114553254 None] Delete group completed successfully. Sep 17 21:24:17.816785 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:17.852350 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (11e1d004-827c-4079-b563-a12833130303) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:17.859050 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2816a896-03d8-4fa5-bd3b-757baf45e4b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:17.876204 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2816a896-03d8-4fa5-bd3b-757baf45e4b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:18Z,deleted=False,deleted_at=None,display_description='',display_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f631d440-70da-4e51-9e54-eba9b08d1bb7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b99d2a0b4d7e40a38e682fc147e968af',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=303db74d-f9c3-4861-abcb-10785db341f5,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:18Z,use_quota=True,user_id='342e60103c9b45bcb348b6f8bcd686ca',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:17.881488 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (873b75b1-c808-4e65-8e6c-4fc55243bd4a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:17.885984 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (873b75b1-c808-4e65-8e6c-4fc55243bd4a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:17.891182 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b5919c8b-7b57-41c8-8ca0-3edbf95b3f73) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:17.894988 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b5919c8b-7b57-41c8-8ca0-3edbf95b3f73) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'f631d440-70da-4e51-9e54-eba9b08d1bb7', 'volume_name': 'volume-f631d440-70da-4e51-9e54-eba9b08d1bb7', 'volume_size': 1, 'snapshot_id': '303db74d-f9c3-4861-abcb-10785db341f5'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:17.899845 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bcfb31dc-5f00-44e1-a1c6-b08af4b95fc0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:17.903659 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bcfb31dc-5f00-44e1-a1c6-b08af4b95fc0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:17.908207 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4960410a-6cb3-40b4-abc9-c659fc1323b8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:17.909424 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Volume f631d440-70da-4e51-9e54-eba9b08d1bb7: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-f631d440-70da-4e51-9e54-eba9b08d1bb7', 'volume_size': 1, 'snapshot_id': '303db74d-f9c3-4861-abcb-10785db341f5'} Sep 17 21:24:17.929211 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] cloning volumes/volume-4f5efa55-03ee-4386-ad16-0c42d83a2e3b@snapshot-303db74d-f9c3-4861-abcb-10785db341f5 to volume-f631d440-70da-4e51-9e54-eba9b08d1bb7 {{(pid=133409) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 17 21:24:17.929889 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:18.015463 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:18.230486 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:18.230858 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:18.252398 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Marking volume f631d440-70da-4e51-9e54-eba9b08d1bb7 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:24:18.262632 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Copying metadata from snapshot 303db74d-f9c3-4861-abcb-10785db341f5 to f631d440-70da-4e51-9e54-eba9b08d1bb7. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:430}} Sep 17 21:24:18.342662 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4960410a-6cb3-40b4-abc9-c659fc1323b8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f631d440-70da-4e51-9e54-eba9b08d1bb7', 'volume_size': 1, 'snapshot_id': '303db74d-f9c3-4861-abcb-10785db341f5'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:18.348098 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (94486701-2b19-4c63-a01c-c496f79c40a6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:18.368972 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Volume volume-f631d440-70da-4e51-9e54-eba9b08d1bb7 (f631d440-70da-4e51-9e54-eba9b08d1bb7): created successfully Sep 17 21:24:18.392425 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (94486701-2b19-4c63-a01c-c496f79c40a6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:18.413274 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (11e1d004-827c-4079-b563-a12833130303) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:18.452117 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-33949588-8aba-4a4e-b893-632995ff9620 tempest-TestVolumeBootPattern-747170341 None] Created volume successfully. Sep 17 21:24:21.100973 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-cc792950-aaec-4f8e-b297-cfd79ccb4d70 tempest-TestVolumeBootPattern-747170341 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:24:21.914462 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-cc792950-aaec-4f8e-b297-cfd79ccb4d70 tempest-TestVolumeBootPattern-747170341 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.813s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:24:21.915490 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-cc792950-aaec-4f8e-b297-cfd79ccb4d70 tempest-TestVolumeBootPattern-747170341 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f631d440-70da-4e51-9e54-eba9b08d1bb7', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'f631d440-70da-4e51-9e54-eba9b08d1bb7', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:24:21.954228 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-cc792950-aaec-4f8e-b297-cfd79ccb4d70 tempest-TestVolumeBootPattern-747170341 None] Connection info returned from driver {'name': 'volumes/volume-f631d440-70da-4e51-9e54-eba9b08d1bb7', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'f631d440-70da-4e51-9e54-eba9b08d1bb7', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'beab1ea6-8185-4e6c-859d-dcad6518bac8'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:24:22.013062 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-56866dc7-2ec7-48a1-910f-628df52c8e64 req-cc792950-aaec-4f8e-b297-cfd79ccb4d70 tempest-TestVolumeBootPattern-747170341 None] attachment_update completed successfully. Sep 17 21:24:29.996119 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:30.022170 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Flow 'volume_create_manager' (6076b19b-2b39-4543-aff0-7707ff878a6d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:30.029515 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cbcf968e-9517-465d-9c80-e60f42f39003) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:30.046519 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cbcf968e-9517-465d-9c80-e60f42f39003) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasNegativeTestJSON-Volume-1554957763',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b5b49a44-88a2-4254-ae29-05c566ecad21,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='4f4b6b67b148445494265e6ad8651a95',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:30Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:30Z,use_quota=True,user_id='1b3966e053ce4c028783294c23cc7560',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:30.052769 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3bee7349-d2c4-4759-9f7c-6bbbd218f23b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:30.056884 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3bee7349-d2c4-4759-9f7c-6bbbd218f23b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:30.063142 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4f7bcbf8-0cf7-4788-a945-32c19e773015) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:30.067291 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4f7bcbf8-0cf7-4788-a945-32c19e773015) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b5b49a44-88a2-4254-ae29-05c566ecad21', 'volume_name': 'volume-b5b49a44-88a2-4254-ae29-05c566ecad21', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:30.072068 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e4778a58-db16-4454-9a0c-b6c83dbbdfbc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:30.076056 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e4778a58-db16-4454-9a0c-b6c83dbbdfbc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:30.080525 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5dffc91f-d8b0-46f8-bbf0-99a1e3e84d6f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:30.081686 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Volume b5b49a44-88a2-4254-ae29-05c566ecad21: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b5b49a44-88a2-4254-ae29-05c566ecad21', 'volume_size': 1} Sep 17 21:24:30.082094 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] creating volume 'volume-b5b49a44-88a2-4254-ae29-05c566ecad21' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:24:30.082856 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:30.153172 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:30.153582 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:30.157339 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5dffc91f-d8b0-46f8-bbf0-99a1e3e84d6f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b5b49a44-88a2-4254-ae29-05c566ecad21', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:30.161822 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e8aee3e3-8852-4e02-ac80-171b289fd44b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:30.171642 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Volume volume-b5b49a44-88a2-4254-ae29-05c566ecad21 (b5b49a44-88a2-4254-ae29-05c566ecad21): created successfully Sep 17 21:24:30.175159 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e8aee3e3-8852-4e02-ac80-171b289fd44b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:30.180473 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Flow 'volume_create_manager' (6076b19b-2b39-4543-aff0-7707ff878a6d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:30.189405 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-15d8951f-9b5f-4d80-99d2-420f16ef5e47 tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Created volume successfully. Sep 17 21:24:31.939038 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Acquiring lock "cinder-b5b49a44-88a2-4254-ae29-05c566ecad21-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:24:31.952174 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Lock "cinder-b5b49a44-88a2-4254-ae29-05c566ecad21-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:24:31.957449 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=64,cinder:UPDATE=25,cinder:INSERT=9,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:32.016195 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:32.117128 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:24:32.117623 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Volume volume-b5b49a44-88a2-4254-ae29-05c566ecad21 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:24:32.123080 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] deleting rbd volume volume-b5b49a44-88a2-4254-ae29-05c566ecad21 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:24:32.383161 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-29b261aa-9c14-443f-a82f-926a260147bc req-1b49c7fc-ba8a-4173-9069-4133918ceff3 tempest-TestVolumeBootPattern-747170341 None] Terminate volume connection completed successfully. Sep 17 21:24:32.384097 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-29b261aa-9c14-443f-a82f-926a260147bc req-1b49c7fc-ba8a-4173-9069-4133918ceff3 tempest-TestVolumeBootPattern-747170341 None] Deleting attachment beab1ea6-8185-4e6c-859d-dcad6518bac8. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:24:32.497380 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Created reservations ['39b1cbba-8d28-4c2f-ac3d-05879c8ce9f7', '876f83b7-471f-435b-846e-fab0abbe2274', 'd3ab4c6e-5f87-4294-8e17-38b6060e700a', 'cc8edbc2-f386-498e-8a86-c1cfeecd7137'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:32.547896 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-29b261aa-9c14-443f-a82f-926a260147bc req-2fd09769-6ae0-4f10-8342-a42fcbc8ebb7 tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-f631d440-70da-4e51-9e54-eba9b08d1bb7-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:24:32.570469 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-29b261aa-9c14-443f-a82f-926a260147bc req-2fd09769-6ae0-4f10-8342-a42fcbc8ebb7 tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-f631d440-70da-4e51-9e54-eba9b08d1bb7-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:24:32.575356 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:32.620658 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:32.687127 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:24:32.693565 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Deleted volume successfully. Sep 17 21:24:32.712992 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-73494301-fa26-4e75-b4f9-19f2b68ce2eb tempest-VolumeQuotasNegativeTestJSON-1381595135 None] Lock "cinder-b5b49a44-88a2-4254-ae29-05c566ecad21-delete_volume" released by "delete_volume" :: held 0.761s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:24:32.729455 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-29b261aa-9c14-443f-a82f-926a260147bc req-2fd09769-6ae0-4f10-8342-a42fcbc8ebb7 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:32.804980 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-29b261aa-9c14-443f-a82f-926a260147bc req-2fd09769-6ae0-4f10-8342-a42fcbc8ebb7 tempest-TestVolumeBootPattern-747170341 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:24:32.812607 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-29b261aa-9c14-443f-a82f-926a260147bc req-2fd09769-6ae0-4f10-8342-a42fcbc8ebb7 tempest-TestVolumeBootPattern-747170341 None] deleting rbd volume volume-f631d440-70da-4e51-9e54-eba9b08d1bb7 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:24:33.023195 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [req-29b261aa-9c14-443f-a82f-926a260147bc req-2fd09769-6ae0-4f10-8342-a42fcbc8ebb7 tempest-TestVolumeBootPattern-747170341 None] Created reservations ['bab64610-5b77-4cf4-a363-65b210813501', '952febc8-7367-4cf4-b7bc-321b91148d26', '0de73283-1157-44a6-a37a-d4e7808ab007', 'b2141498-c8f1-432d-ad64-76070f432f93'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:33.027417 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=63,cinder:INSERT=10,cinder:UPDATE=27 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:33.082623 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-29b261aa-9c14-443f-a82f-926a260147bc req-2fd09769-6ae0-4f10-8342-a42fcbc8ebb7 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:33.129680 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-29b261aa-9c14-443f-a82f-926a260147bc req-2fd09769-6ae0-4f10-8342-a42fcbc8ebb7 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:33.199208 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [req-29b261aa-9c14-443f-a82f-926a260147bc req-2fd09769-6ae0-4f10-8342-a42fcbc8ebb7 tempest-TestVolumeBootPattern-747170341 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:24:33.208260 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-29b261aa-9c14-443f-a82f-926a260147bc req-2fd09769-6ae0-4f10-8342-a42fcbc8ebb7 tempest-TestVolumeBootPattern-747170341 None] Deleted volume successfully. Sep 17 21:24:33.221901 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-29b261aa-9c14-443f-a82f-926a260147bc req-2fd09769-6ae0-4f10-8342-a42fcbc8ebb7 tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-f631d440-70da-4e51-9e54-eba9b08d1bb7-delete_volume" released by "delete_volume" :: held 0.651s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:24:33.959553 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-95e8e572-b0c9-4680-889e-8abd707b6419 tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-303db74d-f9c3-4861-abcb-10785db341f5-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:24:33.973412 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-95e8e572-b0c9-4680-889e-8abd707b6419 tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-303db74d-f9c3-4861-abcb-10785db341f5-delete_snapshot" acquired by "delete_snapshot" :: waited 0.014s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:24:33.975706 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-95e8e572-b0c9-4680-889e-8abd707b6419 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:34.699895 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-95e8e572-b0c9-4680-889e-8abd707b6419 tempest-TestVolumeBootPattern-747170341 None] Created reservations ['1f17fe9a-500d-466a-a7d1-486cf535d7b4', 'c8050aec-dcba-4b84-a377-1fcd37255ac2', '3bcca6a7-d8da-494c-83ad-086decb8a751', '91fd5616-fdf2-4e07-8439-e9a0a6d41227'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:34.768992 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-95e8e572-b0c9-4680-889e-8abd707b6419 tempest-TestVolumeBootPattern-747170341 None] Delete snapshot completed successfully. Sep 17 21:24:34.784539 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-95e8e572-b0c9-4680-889e-8abd707b6419 tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-303db74d-f9c3-4861-abcb-10785db341f5-delete_snapshot" released by "delete_snapshot" :: held 0.811s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:24:35.172151 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-4f5efa55-03ee-4386-ad16-0c42d83a2e3b-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:24:35.186287 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-4f5efa55-03ee-4386-ad16-0c42d83a2e3b-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:24:35.230168 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:35.323450 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:24:35.332691 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] deleting rbd volume volume-4f5efa55-03ee-4386-ad16-0c42d83a2e3b {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:24:35.606528 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:24:35.656065 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] deleting parent snapshot volume-4f5efa55-03ee-4386-ad16-0c42d83a2e3b.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:24:36.764652 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] Created reservations ['b5dd3a32-a4ac-466e-97ee-10af609a248c', 'd310e0ec-4673-495b-bd65-5b5b9fff2a31', 'ab0e23b5-1c85-4c8a-ae8d-8af3889d4c79', 'eb58218a-b529-41b8-b325-7f03c7fa0a04'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:36.823169 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:36.906916 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:36.976153 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:24:36.982038 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] Deleted volume successfully. Sep 17 21:24:37.003284 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a4389cdd-4e54-42c0-9696-4accafd7828b tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-4f5efa55-03ee-4386-ad16-0c42d83a2e3b-delete_volume" released by "delete_volume" :: held 1.817s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:24:37.977776 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:37.998569 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (bad9d9e4-d115-4f68-8120-6b66623d6e2a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:38.004924 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6e6968b1-f7f6-4c46-a55d-5511d083240c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.022265 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6e6968b1-f7f6-4c46-a55d-5511d083240c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:38Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1167692136',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=181d1f0d-1ec7-49df-8b5b-e3c264fefd29,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b99d2a0b4d7e40a38e682fc147e968af',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:38Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:38Z,use_quota=True,user_id='342e60103c9b45bcb348b6f8bcd686ca',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:38.027360 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (abfd6bc2-191c-4f35-975e-f3c87c54811e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.031926 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (abfd6bc2-191c-4f35-975e-f3c87c54811e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:38.037702 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7ad3c9c2-3c1c-42ed-8eaf-7c8809e0ef24) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.146382 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7ad3c9c2-3c1c-42ed-8eaf-7c8809e0ef24) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '181d1f0d-1ec7-49df-8b5b-e3c264fefd29', 'volume_name': 'volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:38.151529 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c7035833-3ef3-4543-8e70-b7589c23ab9e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.155388 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c7035833-3ef3-4543-8e70-b7589c23ab9e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:38.160027 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (abdfd48a-1162-44d0-8edd-04df757e1d3a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.161145 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Volume 181d1f0d-1ec7-49df-8b5b-e3c264fefd29: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 21:24:38.161725 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Cloning 181d1f0d-1ec7-49df-8b5b-e3c264fefd29 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 21:24:38.162465 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:24:38.167403 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:38.186266 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Flow 'volume_create_manager' (5a027d1e-6a19-4261-8bdc-bb3bc16e42f0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:38.192702 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0f9aadc5-0b87-42f2-8db6-9fb51413edd6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.207816 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.045s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:24:38.215847 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0f9aadc5-0b87-42f2-8db6-9fb51413edd6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:38Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeTypesAccessTest-Volume-963524045',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8a12edfc-5f3b-491f-a119-8744e3b4e6af,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e2aae04e106543b5a0b087dbd0aa4782',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:38Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:38Z,use_quota=True,user_id='b45ba1feae9840ebb52de3c701c6d784',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(cb2de0e4-2889-4a75-841b-e7aac5b52703),volume_type_id=cb2de0e4-2889-4a75-841b-e7aac5b52703)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:38.220967 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (08c19005-6cf3-4621-a026-3f9522fa2b57) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.225019 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (08c19005-6cf3-4621-a026-3f9522fa2b57) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:38.229884 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (acb91900-2751-4752-b309-c02b726ab363) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.233772 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (acb91900-2751-4752-b309-c02b726ab363) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8a12edfc-5f3b-491f-a119-8744e3b4e6af', 'volume_name': 'volume-8a12edfc-5f3b-491f-a119-8744e3b4e6af', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:38.238462 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (12c70c77-e5c9-433a-a6c8-99dd2e0335b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.242585 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (12c70c77-e5c9-433a-a6c8-99dd2e0335b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:38.247397 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ad18b3de-cfbc-4ec3-a429-079c24ff4fc0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.248452 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Volume 8a12edfc-5f3b-491f-a119-8744e3b4e6af: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8a12edfc-5f3b-491f-a119-8744e3b4e6af', 'volume_size': 1} Sep 17 21:24:38.248826 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] creating volume 'volume-8a12edfc-5f3b-491f-a119-8744e3b4e6af' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:24:38.249589 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:38.289668 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 24, 38, 281780)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:24:38.290113 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:24:38.290936 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 21:24:38.296350 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.088s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:24:38.297367 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 21:24:38.304626 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 24, 38, 301552)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:24:38.305044 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:24:38.305497 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 21:24:38.307532 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=56,cinder:INSERT=11,cinder:UPDATE=32,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:38.320987 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:38.392878 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] creating snapshot='volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:24:38.737245 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29.clone_snap' to 'volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:24:38.750516 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] extra_specs: {} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:38.751026 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] extra_specs: {} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:38.755390 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ad18b3de-cfbc-4ec3-a429-079c24ff4fc0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8a12edfc-5f3b-491f-a119-8744e3b4e6af', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:38.760902 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c2a6cd91-41fe-44d3-98d6-0a19c1c2078e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.779638 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Volume volume-8a12edfc-5f3b-491f-a119-8744e3b4e6af (8a12edfc-5f3b-491f-a119-8744e3b4e6af): created successfully Sep 17 21:24:38.784913 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c2a6cd91-41fe-44d3-98d6-0a19c1c2078e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:38.791969 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Flow 'volume_create_manager' (5a027d1e-6a19-4261-8bdc-bb3bc16e42f0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:38.805294 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-24617d98-0949-470a-859c-7777ade86449 tempest-VolumeTypesAccessTest-307503888 None] Created volume successfully. Sep 17 21:24:38.882204 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:24:38.883731 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:38.884094 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:38.900393 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:24:38.901204 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Marking volume 181d1f0d-1ec7-49df-8b5b-e3c264fefd29 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:24:38.914338 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to 181d1f0d-1ec7-49df-8b5b-e3c264fefd29. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 21:24:38.914882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Creating volume glance metadata for volume 181d1f0d-1ec7-49df-8b5b-e3c264fefd29 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 21:24:38.984051 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (abdfd48a-1162-44d0-8edd-04df757e1d3a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:38.988597 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b94fdd4a-3a16-44ad-a282-01da5272e1a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:38.998168 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Volume volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29 (181d1f0d-1ec7-49df-8b5b-e3c264fefd29): created successfully Sep 17 21:24:39.001885 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b94fdd4a-3a16-44ad-a282-01da5272e1a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:39.007460 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (bad9d9e4-d115-4f68-8120-6b66623d6e2a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:39.016731 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-9a957e78-1a69-4e79-ad06-71d14b03f94d tempest-TestVolumeBootPattern-747170341 None] Created volume successfully. Sep 17 21:24:39.498670 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] Acquiring lock "cinder-8a12edfc-5f3b-491f-a119-8744e3b4e6af-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:24:39.511894 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] Lock "cinder-8a12edfc-5f3b-491f-a119-8744e3b4e6af-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:24:39.571569 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:39.632880 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:24:39.633298 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] Volume volume-8a12edfc-5f3b-491f-a119-8744e3b4e6af is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:24:39.638590 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] deleting rbd volume volume-8a12edfc-5f3b-491f-a119-8744e3b4e6af {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:24:39.882641 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1523761231 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1523761231, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:24:39.884299 ncdv-master-1 cinder-volume[133409]: WARNING cinder.quota [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1523761231 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1523761231, it is now deprecated. Please use the default quota class for default quota. Sep 17 21:24:39.889915 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=59,cinder:UPDATE=25,cinder:DELETE=4,cinder:INSERT=12 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:39.909988 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] Created reservations ['56b46f6d-ae7b-4bc8-8a55-66f83df30dd5', '49606a70-a6fa-4e03-954d-8730ff597336', '2702c06f-9637-4374-a731-1f8a17b92242', '6e94cc7b-68dc-44c3-863a-eece25a7ee70'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:39.974792 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:40.014685 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:40.052406 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:24:40.057988 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] Deleted volume successfully. Sep 17 21:24:40.081988 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-d073caa5-a74c-4adc-9bac-ea0e0f13308e tempest-VolumeTypesAccessTest-307503888 None] Lock "cinder-8a12edfc-5f3b-491f-a119-8744e3b4e6af-delete_volume" released by "delete_volume" :: held 0.570s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:24:42.704220 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-230c3804-2461-4081-99d3-ec3a174f1f7a req-18ea3fac-62ed-4cd0-bb41-ba0a6e68e9da tempest-TestVolumeBootPattern-747170341 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:24:43.600040 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-230c3804-2461-4081-99d3-ec3a174f1f7a req-18ea3fac-62ed-4cd0-bb41-ba0a6e68e9da tempest-TestVolumeBootPattern-747170341 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.896s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:24:43.601719 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-230c3804-2461-4081-99d3-ec3a174f1f7a req-18ea3fac-62ed-4cd0-bb41-ba0a6e68e9da tempest-TestVolumeBootPattern-747170341 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '181d1f0d-1ec7-49df-8b5b-e3c264fefd29', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:24:43.633989 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-230c3804-2461-4081-99d3-ec3a174f1f7a req-18ea3fac-62ed-4cd0-bb41-ba0a6e68e9da tempest-TestVolumeBootPattern-747170341 None] Connection info returned from driver {'name': 'volumes/volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '181d1f0d-1ec7-49df-8b5b-e3c264fefd29', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '80f26435-5f31-4392-bf40-d91ad1649289'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:24:43.735233 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-230c3804-2461-4081-99d3-ec3a174f1f7a req-18ea3fac-62ed-4cd0-bb41-ba0a6e68e9da tempest-TestVolumeBootPattern-747170341 None] attachment_update completed successfully. Sep 17 21:24:46.639855 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:46.665154 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Flow 'volume_create_manager' (1feadf38-b04b-400a-a572-40dc711f2709) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:46.673444 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1228dd9a-19f2-43d4-b539-746d8467eca6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:46.698423 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1228dd9a-19f2-43d4-b539-746d8467eca6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-2089365635',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=02ed1f85-21fe-40c2-89e1-3b1ee79a2a70,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='83353167d11c4133a80f5630e7b24f0a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:47Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:47Z,use_quota=True,user_id='f8a3a7c83aa14e088423d8dfbe39ff19',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:46.704018 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9dbf2ffe-a56d-41fd-adeb-0e523b4393e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:46.708150 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9dbf2ffe-a56d-41fd-adeb-0e523b4393e1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:46.713353 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ac6576dd-2927-470d-82bd-c16e11ed3434) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:46.717275 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ac6576dd-2927-470d-82bd-c16e11ed3434) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '02ed1f85-21fe-40c2-89e1-3b1ee79a2a70', 'volume_name': 'volume-02ed1f85-21fe-40c2-89e1-3b1ee79a2a70', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:46.722169 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1da3387a-fd67-49df-8be4-5fc831ff3938) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:46.726024 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1da3387a-fd67-49df-8be4-5fc831ff3938) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:46.730566 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3fe05eb2-c835-4ace-9dff-fa844690bdd0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:46.731626 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Volume 02ed1f85-21fe-40c2-89e1-3b1ee79a2a70: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-02ed1f85-21fe-40c2-89e1-3b1ee79a2a70', 'volume_size': 1} Sep 17 21:24:46.732047 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] creating volume 'volume-02ed1f85-21fe-40c2-89e1-3b1ee79a2a70' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:24:46.733160 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:46.800225 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:46.800914 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:46.804910 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3fe05eb2-c835-4ace-9dff-fa844690bdd0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-02ed1f85-21fe-40c2-89e1-3b1ee79a2a70', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:46.809586 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (920e2334-ef0c-4d65-8d97-408b21f779d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:46.819965 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Volume volume-02ed1f85-21fe-40c2-89e1-3b1ee79a2a70 (02ed1f85-21fe-40c2-89e1-3b1ee79a2a70): created successfully Sep 17 21:24:46.825992 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (920e2334-ef0c-4d65-8d97-408b21f779d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:46.831955 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Flow 'volume_create_manager' (1feadf38-b04b-400a-a572-40dc711f2709) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:46.842517 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-01d32c4e-6bb0-4cd8-8ea7-0ea137836d79 tempest-VolumesListAdminTestJSON-147435163 None] Created volume successfully. Sep 17 21:24:48.084493 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:48.104591 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Flow 'volume_create_manager' (9567a6dc-3ba7-4104-998a-7b56166458d2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:48.111319 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b3b20773-2c06-4c3d-9072-a05c401b44c1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:48.128569 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b3b20773-2c06-4c3d-9072-a05c401b44c1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-1911031701',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=0051b21c-74f9-46db-859b-61a7b8acb95f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='83353167d11c4133a80f5630e7b24f0a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:48Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:48Z,use_quota=True,user_id='f8a3a7c83aa14e088423d8dfbe39ff19',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:48.134084 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3917583f-2624-48af-a877-ee6cecbb0056) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:48.138464 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3917583f-2624-48af-a877-ee6cecbb0056) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:48.143388 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6155f18c-2332-4fd5-9034-16b221db66a5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:48.147400 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6155f18c-2332-4fd5-9034-16b221db66a5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '0051b21c-74f9-46db-859b-61a7b8acb95f', 'volume_name': 'volume-0051b21c-74f9-46db-859b-61a7b8acb95f', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:48.152125 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (25552e08-b53f-4abd-8fc8-602f85f6d630) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:48.155889 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (25552e08-b53f-4abd-8fc8-602f85f6d630) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:48.161337 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8e46095e-a314-4651-8587-ff6ba13b10c1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:48.162436 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Volume 0051b21c-74f9-46db-859b-61a7b8acb95f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-0051b21c-74f9-46db-859b-61a7b8acb95f', 'volume_size': 1} Sep 17 21:24:48.162824 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] creating volume 'volume-0051b21c-74f9-46db-859b-61a7b8acb95f' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:24:48.163476 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:48.226656 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:48.227135 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:48.231007 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8e46095e-a314-4651-8587-ff6ba13b10c1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-0051b21c-74f9-46db-859b-61a7b8acb95f', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:48.236274 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (56376b72-1318-4016-95af-615624db8127) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:48.247328 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Volume volume-0051b21c-74f9-46db-859b-61a7b8acb95f (0051b21c-74f9-46db-859b-61a7b8acb95f): created successfully Sep 17 21:24:48.253662 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (56376b72-1318-4016-95af-615624db8127) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:48.262842 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Flow 'volume_create_manager' (9567a6dc-3ba7-4104-998a-7b56166458d2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:48.274204 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7ebcfc77-57c8-4a26-825a-937344eec21d tempest-VolumesListAdminTestJSON-147435163 None] Created volume successfully. Sep 17 21:24:49.552256 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:49.574094 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Flow 'volume_create_manager' (2367a2b5-d21b-4b90-9195-3be06eaa1c56) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:49.580263 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f46a1bc2-9d5f-432e-b114-60e30ff76bf7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:49.596156 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f46a1bc2-9d5f-432e-b114-60e30ff76bf7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:49Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-235271279',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=35fcbb7a-ddc4-4a93-9aa1-b03268f47aad,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='83353167d11c4133a80f5630e7b24f0a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:50Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:50Z,use_quota=True,user_id='f8a3a7c83aa14e088423d8dfbe39ff19',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:49.601425 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6c54614e-71a8-416a-8b6c-9eecc59ae812) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:49.605382 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6c54614e-71a8-416a-8b6c-9eecc59ae812) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:49.610220 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d2c4a23d-3f91-453e-8bf8-658a4cd9a8af) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:49.614462 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d2c4a23d-3f91-453e-8bf8-658a4cd9a8af) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '35fcbb7a-ddc4-4a93-9aa1-b03268f47aad', 'volume_name': 'volume-35fcbb7a-ddc4-4a93-9aa1-b03268f47aad', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:49.619485 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (da5bdaf9-3be1-4926-9f9c-b9b2b1417490) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:49.623256 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (da5bdaf9-3be1-4926-9f9c-b9b2b1417490) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:49.627957 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e5931148-5c87-48d4-88fe-f91ce929c9e6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:49.629144 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Volume 35fcbb7a-ddc4-4a93-9aa1-b03268f47aad: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-35fcbb7a-ddc4-4a93-9aa1-b03268f47aad', 'volume_size': 1} Sep 17 21:24:49.629702 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] creating volume 'volume-35fcbb7a-ddc4-4a93-9aa1-b03268f47aad' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:24:49.630349 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:49.709879 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:49.710913 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:49.715501 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e5931148-5c87-48d4-88fe-f91ce929c9e6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-35fcbb7a-ddc4-4a93-9aa1-b03268f47aad', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:49.721253 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d8e465fc-3520-4aa4-8797-cb4c3a9b4dcb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:49.738179 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Volume volume-35fcbb7a-ddc4-4a93-9aa1-b03268f47aad (35fcbb7a-ddc4-4a93-9aa1-b03268f47aad): created successfully Sep 17 21:24:49.743040 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d8e465fc-3520-4aa4-8797-cb4c3a9b4dcb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:49.748862 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Flow 'volume_create_manager' (2367a2b5-d21b-4b90-9195-3be06eaa1c56) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:49.758344 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7d4f0e21-a3a5-45e9-837f-b5958f6654b4 tempest-VolumesListAdminTestJSON-147435163 None] Created volume successfully. Sep 17 21:24:51.056599 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:51.076779 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Flow 'volume_create_manager' (dab962e8-3eca-4c12-9905-f51b4a63c789) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:51.083410 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8a923bed-7748-49b3-ae42-63ff7d8f7d61) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:51.099713 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8a923bed-7748-49b3-ae42-63ff7d8f7d61) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:51Z,deleted=False,deleted_at=None,display_description=None,display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5afcc0b6-dec3-479e-9890-69236c6eb21a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='68a7dfc29e984352ad11519b0e2b69f7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:51Z,use_quota=True,user_id='92b99b8f12624271b29c9af71272e6ca',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:51.104974 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (653bb091-291d-46ce-a523-febde7997512) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:51.108985 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (653bb091-291d-46ce-a523-febde7997512) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:51.113918 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a33237a1-b287-42c8-9479-08be9606042d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:51.118347 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a33237a1-b287-42c8-9479-08be9606042d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5afcc0b6-dec3-479e-9890-69236c6eb21a', 'volume_name': 'volume-5afcc0b6-dec3-479e-9890-69236c6eb21a', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:51.123174 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (516d6456-a9f5-47d5-a386-de90212b2c38) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:51.126870 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (516d6456-a9f5-47d5-a386-de90212b2c38) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:51.131382 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (953a3c0e-11df-4328-9da7-b808cd1ddd5f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:51.132428 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Volume 5afcc0b6-dec3-479e-9890-69236c6eb21a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5afcc0b6-dec3-479e-9890-69236c6eb21a', 'volume_size': 1} Sep 17 21:24:51.132816 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] creating volume 'volume-5afcc0b6-dec3-479e-9890-69236c6eb21a' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:24:51.133472 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:51.185222 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:51.185635 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:51.189451 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (953a3c0e-11df-4328-9da7-b808cd1ddd5f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5afcc0b6-dec3-479e-9890-69236c6eb21a', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:51.193986 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d2675c92-49c1-461b-98c9-2e7b6c8667ce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:51.206632 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Volume volume-5afcc0b6-dec3-479e-9890-69236c6eb21a (5afcc0b6-dec3-479e-9890-69236c6eb21a): created successfully Sep 17 21:24:51.211133 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d2675c92-49c1-461b-98c9-2e7b6c8667ce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:51.217060 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Flow 'volume_create_manager' (dab962e8-3eca-4c12-9905-f51b4a63c789) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:51.226972 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-58e3bfe4-5b4f-4495-87bc-5992a0b44b56 tempest-VolumesListAdminTestJSON-977712474 None] Created volume successfully. Sep 17 21:24:52.184627 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] Acquiring lock "cinder-5afcc0b6-dec3-479e-9890-69236c6eb21a-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:24:52.196742 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] Lock "cinder-5afcc0b6-dec3-479e-9890-69236c6eb21a-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:24:52.229868 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:52.301933 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:24:52.302963 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] Volume volume-5afcc0b6-dec3-479e-9890-69236c6eb21a is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:24:52.310979 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] deleting rbd volume volume-5afcc0b6-dec3-479e-9890-69236c6eb21a {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:24:52.318103 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] Acquiring lock "cinder-35fcbb7a-ddc4-4a93-9aa1-b03268f47aad-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:24:52.332473 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] Lock "cinder-35fcbb7a-ddc4-4a93-9aa1-b03268f47aad-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:24:52.379649 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:52.476203 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:24:52.476572 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] Volume volume-35fcbb7a-ddc4-4a93-9aa1-b03268f47aad is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:24:52.488050 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] deleting rbd volume volume-35fcbb7a-ddc4-4a93-9aa1-b03268f47aad {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:24:52.661355 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] Created reservations ['37488aa9-c079-4a8e-b42f-b1ef013d1faf', '3272f4ae-7e30-4dc8-901b-5abdb20a05ea', '8e4c87f0-db1a-4104-aacb-dae47292ab62', '6b024495-c495-465f-8914-0676396ed61b'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:52.734160 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:52.785925 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:52.821006 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:24:52.833132 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] Deleted volume successfully. Sep 17 21:24:52.837010 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=62,cinder:UPDATE=26,cinder:INSERT=9,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:52.860002 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-6a749f07-0507-483a-b53b-5e12ce0ce83c tempest-VolumesListAdminTestJSON-977712474 None] Lock "cinder-5afcc0b6-dec3-479e-9890-69236c6eb21a-delete_volume" released by "delete_volume" :: held 0.663s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:24:52.878793 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] Created reservations ['b087b9a6-e0bf-418d-81ed-36de7d198dd4', '1dab6d96-fc02-41f5-9391-63cdff664250', 'e52fade9-0e60-47bc-a55f-b192929757b0', '1aafb47d-5341-4106-be20-44bcdd323db2'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:52.934045 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:52.991117 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:53.030177 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:24:53.040038 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] Deleted volume successfully. Sep 17 21:24:53.054339 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-ee238e1f-7ed4-4cf3-b219-6b73c3984920 tempest-VolumesListAdminTestJSON-147435163 None] Lock "cinder-35fcbb7a-ddc4-4a93-9aa1-b03268f47aad-delete_volume" released by "delete_volume" :: held 0.722s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:24:53.441018 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] Acquiring lock "cinder-0051b21c-74f9-46db-859b-61a7b8acb95f-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:24:53.452690 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] Lock "cinder-0051b21c-74f9-46db-859b-61a7b8acb95f-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:24:53.506859 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:53.546947 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:24:53.547309 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] Volume volume-0051b21c-74f9-46db-859b-61a7b8acb95f is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:24:53.555249 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] deleting rbd volume volume-0051b21c-74f9-46db-859b-61a7b8acb95f {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:24:53.835757 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=30,cinder:SELECT=61,cinder:DELETE=4,cinder:INSERT=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:24:53.868467 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] Created reservations ['3910de6b-ed6e-482a-91c2-7adbcbd82463', '7a2d84fc-737e-4eee-a894-0d49d0c84d96', '1253091b-41e1-481e-8255-9366eb6801a0', 'bd2c8f72-77ef-4d04-be8e-9242367107c6'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:53.932728 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:53.966390 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:54.027501 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:24:54.033269 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] Deleted volume successfully. Sep 17 21:24:54.049960 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-12fe3d6b-e9b8-4ab2-bd1a-3ed88eea93e9 tempest-VolumesListAdminTestJSON-147435163 None] Lock "cinder-0051b21c-74f9-46db-859b-61a7b8acb95f-delete_volume" released by "delete_volume" :: held 0.597s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:24:54.589083 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-5b8f0207-1f4d-4698-979a-19374f40641e req-f26c9f99-3d28-491a-974f-21e41d56179d tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:54.717429 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] Acquiring lock "cinder-02ed1f85-21fe-40c2-89e1-3b1ee79a2a70-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:24:54.732226 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] Lock "cinder-02ed1f85-21fe-40c2-89e1-3b1ee79a2a70-delete_volume" acquired by "delete_volume" :: waited 0.015s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:24:54.769670 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:54.823570 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:24:54.824001 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] Volume volume-02ed1f85-21fe-40c2-89e1-3b1ee79a2a70 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:24:54.830366 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] deleting rbd volume volume-02ed1f85-21fe-40c2-89e1-3b1ee79a2a70 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:24:55.141103 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] Created reservations ['ddf40c3f-6c2e-40e9-8419-8f4c0d1691fe', 'cb1cb61f-16cc-44da-9414-8de00d6b7de1', '78749f1d-2b2b-4818-a4f4-f60d9b64deb0', '7820d220-7876-4547-a66f-4f1cf0e9c74e'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:24:55.218783 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:55.288967 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:55.340592 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:24:55.344944 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] Deleted volume successfully. Sep 17 21:24:55.368494 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2907c47c-be11-4d32-8d9f-fa61fca19587 tempest-VolumesListAdminTestJSON-147435163 None] Lock "cinder-02ed1f85-21fe-40c2-89e1-3b1ee79a2a70-delete_volume" released by "delete_volume" :: held 0.636s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:24:55.779360 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-5b8f0207-1f4d-4698-979a-19374f40641e req-f26c9f99-3d28-491a-974f-21e41d56179d tempest-TestVolumeBootPattern-747170341 None] Create snapshot completed successfully Sep 17 21:24:58.771964 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:58.793833 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Flow 'volume_create_manager' (a33ab665-7a07-44d7-8d15-5a10e7b170a3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:58.801173 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cce535b2-9a26-4f4f-a95a-2c4ed8941b85) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:58.817827 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cce535b2-9a26-4f4f-a95a-2c4ed8941b85) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesDeleteCascade-Volume-1537574783',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5998f7ec-5a4a-4833-af88-32d15eb628f7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='454be92019f0498980bebad9e988f37e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:59Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:59Z,use_quota=True,user_id='0b32996144954c8face21e641875f7a0',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:58.823335 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2eea0935-8530-408d-9b9b-565a4f8d86f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:58.828852 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2eea0935-8530-408d-9b9b-565a4f8d86f9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:58.834574 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ba86992f-31b0-405a-8dc9-e82812135cdf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:58.839235 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ba86992f-31b0-405a-8dc9-e82812135cdf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5998f7ec-5a4a-4833-af88-32d15eb628f7', 'volume_name': 'volume-5998f7ec-5a4a-4833-af88-32d15eb628f7', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:58.844052 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (59a2437d-b9db-4f99-a079-a7b406aa62a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:58.848098 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (59a2437d-b9db-4f99-a079-a7b406aa62a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:58.852691 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (01b10092-ba3d-4dd7-9167-115ed3f5568a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:58.854026 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Volume 5998f7ec-5a4a-4833-af88-32d15eb628f7: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5998f7ec-5a4a-4833-af88-32d15eb628f7', 'volume_size': 1} Sep 17 21:24:58.854589 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] creating volume 'volume-5998f7ec-5a4a-4833-af88-32d15eb628f7' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:24:58.855936 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:58.951863 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:58.952307 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:58.956660 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (01b10092-ba3d-4dd7-9167-115ed3f5568a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5998f7ec-5a4a-4833-af88-32d15eb628f7', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:58.971948 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (37e9ba93-fafd-43c5-8097-0bb8c43793b2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:58.985783 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Volume volume-5998f7ec-5a4a-4833-af88-32d15eb628f7 (5998f7ec-5a4a-4833-af88-32d15eb628f7): created successfully Sep 17 21:24:58.990011 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (37e9ba93-fafd-43c5-8097-0bb8c43793b2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:58.995890 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Flow 'volume_create_manager' (a33ab665-7a07-44d7-8d15-5a10e7b170a3) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:59.007933 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-39547ad6-1190-4e4c-bf40-0ddd57dea715 tempest-VolumesDeleteCascade-78755505 None] Created volume successfully. Sep 17 21:24:59.489789 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:24:59.520371 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (f246df19-8f9b-464f-962f-6e36b19baccc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:24:59.526739 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2110416f-0378-4628-bc6b-0302abfba669) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:59.543830 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2110416f-0378-4628-bc6b-0302abfba669) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:24:59Z,deleted=False,deleted_at=None,display_description='',display_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4459bb90-d891-42ee-a973-a12bb406a773,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b99d2a0b4d7e40a38e682fc147e968af',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:24:59Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=ebc81b3f-de04-4713-a1ca-0d2e7f8eb20f,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:24:59Z,use_quota=True,user_id='342e60103c9b45bcb348b6f8bcd686ca',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:59.549130 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (060e7acc-c971-4f85-a258-6914910d196b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:59.553172 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (060e7acc-c971-4f85-a258-6914910d196b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:59.558017 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e712173a-90c5-4f5c-9fb7-40687d3d3c18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:59.561855 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e712173a-90c5-4f5c-9fb7-40687d3d3c18) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '4459bb90-d891-42ee-a973-a12bb406a773', 'volume_name': 'volume-4459bb90-d891-42ee-a973-a12bb406a773', 'volume_size': 1, 'snapshot_id': 'ebc81b3f-de04-4713-a1ca-0d2e7f8eb20f'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:59.566515 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4215d3f8-0742-474b-a00b-5a3df0526f7a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:59.570881 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4215d3f8-0742-474b-a00b-5a3df0526f7a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:59.575516 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (db9a7c8c-fc0b-4c6a-bfa8-84b15ec56c6a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:59.576763 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Volume 4459bb90-d891-42ee-a973-a12bb406a773: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-4459bb90-d891-42ee-a973-a12bb406a773', 'volume_size': 1, 'snapshot_id': 'ebc81b3f-de04-4713-a1ca-0d2e7f8eb20f'} Sep 17 21:24:59.604503 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] cloning volumes/volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29@snapshot-ebc81b3f-de04-4713-a1ca-0d2e7f8eb20f to volume-4459bb90-d891-42ee-a973-a12bb406a773 {{(pid=133409) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 17 21:24:59.605260 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:59.685906 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:24:59.872922 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:24:59.874872 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:24:59.901365 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Marking volume 4459bb90-d891-42ee-a973-a12bb406a773 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:24:59.915036 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Copying metadata from snapshot ebc81b3f-de04-4713-a1ca-0d2e7f8eb20f to 4459bb90-d891-42ee-a973-a12bb406a773. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:430}} Sep 17 21:24:59.976546 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (db9a7c8c-fc0b-4c6a-bfa8-84b15ec56c6a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4459bb90-d891-42ee-a973-a12bb406a773', 'volume_size': 1, 'snapshot_id': 'ebc81b3f-de04-4713-a1ca-0d2e7f8eb20f'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:24:59.989113 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c7aa1627-78c6-492f-ab79-ceb4a65cc46b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:24:59.995146 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:INSERT=18,cinder:SELECT=55,cinder:UPDATE=24,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:00.005127 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Volume volume-4459bb90-d891-42ee-a973-a12bb406a773 (4459bb90-d891-42ee-a973-a12bb406a773): created successfully Sep 17 21:25:00.017643 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c7aa1627-78c6-492f-ab79-ceb4a65cc46b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:00.028893 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (f246df19-8f9b-464f-962f-6e36b19baccc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:25:00.049325 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-ada0261c-e510-4663-97d9-a859bbbc23db tempest-TestVolumeBootPattern-747170341 None] Created volume successfully. Sep 17 21:25:00.123240 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-79673eab-76b5-46bf-b44d-251b8437f14b tempest-VolumesDeleteCascade-78755505 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:00.930506 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-79673eab-76b5-46bf-b44d-251b8437f14b tempest-VolumesDeleteCascade-78755505 None] Create snapshot completed successfully Sep 17 21:25:01.035189 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:25:01.035790 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:01.087344 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:01.165663 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:25:01.505560 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-7d708482-9f12-4a9a-b716-1eabfd8c59c0 tempest-VolumesDeleteCascade-78755505 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:02.739637 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-7d708482-9f12-4a9a-b716-1eabfd8c59c0 tempest-VolumesDeleteCascade-78755505 None] Create snapshot completed successfully Sep 17 21:25:02.895974 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-1391a48a-6688-4cac-a043-1162dcd732fe req-9b9d3ce6-a395-409f-800d-a3bf6de5c7a6 tempest-TestVolumeBootPattern-747170341 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:25:03.688975 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Acquiring lock "cinder-5998f7ec-5a4a-4833-af88-32d15eb628f7-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:25:03.726876 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Lock "cinder-5998f7ec-5a4a-4833-af88-32d15eb628f7-delete_volume" acquired by "delete_volume" :: waited 0.052s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:25:03.819939 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=29,cinder:SELECT=53,cinder:DELETE=3,cinder:INSERT=15 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:03.831741 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Performing cascade delete. {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/manager.py:1008}} Sep 17 21:25:03.882597 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Acquiring lock "cinder-140717a3-3535-4fc5-b756-f70164d15df7-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:25:03.896276 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Lock "cinder-140717a3-3535-4fc5-b756-f70164d15df7-delete_snapshot" acquired by "delete_snapshot" :: waited 0.014s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:25:03.915130 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:04.030305 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-1391a48a-6688-4cac-a043-1162dcd732fe req-9b9d3ce6-a395-409f-800d-a3bf6de5c7a6 tempest-TestVolumeBootPattern-747170341 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.134s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:25:04.030955 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-1391a48a-6688-4cac-a043-1162dcd732fe req-9b9d3ce6-a395-409f-800d-a3bf6de5c7a6 tempest-TestVolumeBootPattern-747170341 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-4459bb90-d891-42ee-a973-a12bb406a773', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '4459bb90-d891-42ee-a973-a12bb406a773', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:25:04.063105 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-9b9d3ce6-a395-409f-800d-a3bf6de5c7a6 tempest-TestVolumeBootPattern-747170341 None] Connection info returned from driver {'name': 'volumes/volume-4459bb90-d891-42ee-a973-a12bb406a773', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '4459bb90-d891-42ee-a973-a12bb406a773', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '7aca61b3-0bfb-4bc5-b183-24301e363755'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:25:04.141432 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-1391a48a-6688-4cac-a043-1162dcd732fe req-9b9d3ce6-a395-409f-800d-a3bf6de5c7a6 tempest-TestVolumeBootPattern-747170341 None] attachment_update completed successfully. Sep 17 21:25:04.792981 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Created reservations ['3151fe1b-4930-42fd-b972-ffbfe585c4ea', '455b7132-5cac-4f6c-b325-9a2693594e0a', '60ec14a9-2db8-44fc-a6ba-4c7ddddd4817', 'c6667418-31bb-4969-ac3e-6cbb46ddbdda'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:25:04.863744 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Delete snapshot completed successfully. Sep 17 21:25:04.869636 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Lock "cinder-140717a3-3535-4fc5-b756-f70164d15df7-delete_snapshot" released by "delete_snapshot" :: held 0.973s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:25:04.872228 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Acquiring lock "cinder-fda9a651-54c6-4971-8f57-f558644c5fa3-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:25:04.882467 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Lock "cinder-fda9a651-54c6-4971-8f57-f558644c5fa3-delete_snapshot" acquired by "delete_snapshot" :: waited 0.011s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:25:04.899087 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:05.799361 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Created reservations ['e2b71a4d-aba5-45ba-b657-8491f1c3b862', 'bb5eb2db-a3a2-4621-9a99-15b1de29f6fa', '123a6c92-739f-4e8b-b0d8-16f6b0193d26', '561d2d2a-20b7-441c-9429-5999702723c4'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:25:05.838666 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Delete snapshot completed successfully. Sep 17 21:25:05.844079 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Lock "cinder-fda9a651-54c6-4971-8f57-f558644c5fa3-delete_snapshot" released by "delete_snapshot" :: held 0.962s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:25:05.844850 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Snapshots deleted, issuing volume delete {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/manager.py:1023}} Sep 17 21:25:05.845941 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:05.898013 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:25:05.898590 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Volume volume-5998f7ec-5a4a-4833-af88-32d15eb628f7 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:25:05.912133 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] deleting rbd volume volume-5998f7ec-5a4a-4833-af88-32d15eb628f7 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:25:06.099219 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Created reservations ['44c4a1ac-7c5e-4931-823e-86e60705081a', '44c72b89-abb0-4d2d-b634-e47eff29cd8b', '0cfb2778-de0e-4057-9a73-78d662132521', '0ef80572-592b-4e6a-aa81-db0cdc9b5b8b'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:25:06.151494 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:06.182402 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:06.220028 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:25:06.226087 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Deleted volume successfully. Sep 17 21:25:06.242872 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-59f9fb75-909e-4a98-b706-df33280379b0 tempest-VolumesDeleteCascade-78755505 None] Lock "cinder-5998f7ec-5a4a-4833-af88-32d15eb628f7-delete_volume" released by "delete_volume" :: held 2.517s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:25:11.376620 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:25:11.429102 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Flow 'volume_create_manager' (e8bdf38e-3581-4a53-b81b-682cda95d8cb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:25:11.462661 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (42ac3bf2-d13a-440b-b116-c8e0be40c89d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:11.486574 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (42ac3bf2-d13a-440b-b116-c8e0be40c89d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:25:11Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersTest-Volume-226132225',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=7cffd631-d72d-4e7a-8231-432d95901dc0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='120e67d0734d420c90eec58239f7c907',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:25:11Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:25:11Z,use_quota=True,user_id='677033f05fce4c16bcb573ec22f27fab',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:11.494439 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (33d1680b-4c8e-4aa8-87a6-b4320c33b11c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:11.499678 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (33d1680b-4c8e-4aa8-87a6-b4320c33b11c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:11.505409 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ae770228-f809-4edd-90bd-7e2110aa49a5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:11.510642 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ae770228-f809-4edd-90bd-7e2110aa49a5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '7cffd631-d72d-4e7a-8231-432d95901dc0', 'volume_name': 'volume-7cffd631-d72d-4e7a-8231-432d95901dc0', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:11.516424 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (01890286-832b-4951-a2cd-33a73f836b56) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:11.523229 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (01890286-832b-4951-a2cd-33a73f836b56) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:11.534055 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6e47a15d-aec2-4573-9684-7d0399c0e13d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:11.535570 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Volume 7cffd631-d72d-4e7a-8231-432d95901dc0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-7cffd631-d72d-4e7a-8231-432d95901dc0', 'volume_size': 1} Sep 17 21:25:11.536127 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] creating volume 'volume-7cffd631-d72d-4e7a-8231-432d95901dc0' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:25:11.537390 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:11.658367 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:25:11.659161 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:25:11.667591 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6e47a15d-aec2-4573-9684-7d0399c0e13d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7cffd631-d72d-4e7a-8231-432d95901dc0', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:11.674096 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (969c05f8-51ec-40fa-96f6-989e0ddbee30) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:11.699511 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Volume volume-7cffd631-d72d-4e7a-8231-432d95901dc0 (7cffd631-d72d-4e7a-8231-432d95901dc0): created successfully Sep 17 21:25:11.703524 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (969c05f8-51ec-40fa-96f6-989e0ddbee30) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:11.710866 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Flow 'volume_create_manager' (e8bdf38e-3581-4a53-b81b-682cda95d8cb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:25:11.721535 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-69730f5c-b4a0-445c-8aad-78e6b6265468 tempest-VolumesTransfersTest-1329796951 None] Created volume successfully. Sep 17 21:25:13.426009 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-3126ee88-881b-4471-9bbb-f28e0534eda3 tempest-VolumesTransfersTest-501265175 None] Transfer volume completed successfully. Sep 17 21:25:14.142069 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] Acquiring lock "cinder-7cffd631-d72d-4e7a-8231-432d95901dc0-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:25:14.165537 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] Lock "cinder-7cffd631-d72d-4e7a-8231-432d95901dc0-delete_volume" acquired by "delete_volume" :: waited 0.023s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:25:14.217202 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=64,cinder:UPDATE=26,cinder:DELETE=3,cinder:INSERT=7 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:14.220615 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:14.296619 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:25:14.297157 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] Volume volume-7cffd631-d72d-4e7a-8231-432d95901dc0 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:25:14.308014 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] deleting rbd volume volume-7cffd631-d72d-4e7a-8231-432d95901dc0 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:25:14.648071 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] Created reservations ['01a3d877-b0b4-43a1-867e-d8fdc19c9500', '0c49081c-548f-413f-90ef-404e67c801a3', '38f6542c-468f-4a9f-986f-c76e2f64efd8', '87287313-2842-4f87-94fc-5b1ea44e8d73'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:25:14.715526 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:14.787359 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:14.856957 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:25:14.863158 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] Deleted volume successfully. Sep 17 21:25:14.893732 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-4686bb38-1239-41b2-a4d1-7c1aff6d77a5 tempest-VolumesTransfersTest-388601838 None] Lock "cinder-7cffd631-d72d-4e7a-8231-432d95901dc0-delete_volume" released by "delete_volume" :: held 0.728s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:25:15.887561 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:25:15.926501 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Flow 'volume_create_manager' (ce38ce57-0969-4830-914f-b2d01deefafa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:25:15.934783 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (19472ee9-7b21-4be0-8859-20e4c093bede) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:15.953712 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (19472ee9-7b21-4be0-8859-20e4c093bede) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:25:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersTest-Volume-878611978',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='120e67d0734d420c90eec58239f7c907',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:25:16Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:25:16Z,use_quota=True,user_id='677033f05fce4c16bcb573ec22f27fab',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:15.962267 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9a1b3ba7-ee1d-4da7-9fd4-07327d8b83d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:15.970270 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9a1b3ba7-ee1d-4da7-9fd4-07327d8b83d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:15.977487 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bf53392d-4233-4e9e-a06c-805a329504df) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:15.982338 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bf53392d-4233-4e9e-a06c-805a329504df) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24', 'volume_name': 'volume-ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:15.988644 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (aa1e9eba-a770-49b2-9dbe-456070bb62bd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:15.996739 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (aa1e9eba-a770-49b2-9dbe-456070bb62bd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:16.002509 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1c6021c8-9683-4789-94e9-a7be71075279) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:16.004164 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Volume ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24', 'volume_size': 1} Sep 17 21:25:16.004719 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] creating volume 'volume-ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:25:16.005928 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:16.134045 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:25:16.134632 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:25:16.139392 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1c6021c8-9683-4789-94e9-a7be71075279) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:16.145195 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (199c8426-3533-4116-b416-96c88ea661c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:16.162276 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Volume volume-ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24 (ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24): created successfully Sep 17 21:25:16.169737 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (199c8426-3533-4116-b416-96c88ea661c5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:16.177163 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Flow 'volume_create_manager' (ce38ce57-0969-4830-914f-b2d01deefafa) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:25:16.195931 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c11cbff3-6a19-4737-9734-146589c7e47d tempest-VolumesTransfersTest-1329796951 None] Created volume successfully. Sep 17 21:25:17.323264 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] Acquiring lock "cinder-ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:25:17.339641 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] Lock "cinder-ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:25:17.359677 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=59,cinder:UPDATE=32,cinder:INSERT=7,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:17.465957 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:17.549710 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:25:17.550269 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] Volume volume-ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:25:17.556316 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] deleting rbd volume volume-ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:25:17.992943 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] Created reservations ['39574f6a-50f7-4943-af54-a5d6918b8b38', 'e70a004e-a552-46d6-9de4-338f0dc28c2f', 'ec0db457-4085-4f82-871b-19f6a42aa937', '92d174aa-f474-47ad-bb02-bbd39b625aed'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:25:18.081728 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:18.168536 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:18.265754 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:25:18.273729 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] Deleted volume successfully. Sep 17 21:25:18.297241 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-58782473-8925-44f2-aac4-72f2222c72b4 tempest-VolumesTransfersTest-388601838 None] Lock "cinder-ad43a3cd-61ae-40bc-ba3a-d9a804ec7b24-delete_volume" released by "delete_volume" :: held 0.958s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:25:18.375184 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-c91eb2f9-089e-42d9-bcd3-82249c97bf9c tempest-TestVolumeBootPattern-747170341 None] Terminate volume connection completed successfully. Sep 17 21:25:18.376266 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-c91eb2f9-089e-42d9-bcd3-82249c97bf9c tempest-TestVolumeBootPattern-747170341 None] Deleting attachment 7aca61b3-0bfb-4bc5-b183-24301e363755. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:25:18.602782 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-f45c96ec-2b65-4398-b6fe-ef0346466b9c tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-4459bb90-d891-42ee-a973-a12bb406a773-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:25:18.623464 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-f45c96ec-2b65-4398-b6fe-ef0346466b9c tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-4459bb90-d891-42ee-a973-a12bb406a773-delete_volume" acquired by "delete_volume" :: waited 0.021s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:25:18.692214 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-f45c96ec-2b65-4398-b6fe-ef0346466b9c tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:18.835751 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-f45c96ec-2b65-4398-b6fe-ef0346466b9c tempest-TestVolumeBootPattern-747170341 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:25:18.848965 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-f45c96ec-2b65-4398-b6fe-ef0346466b9c tempest-TestVolumeBootPattern-747170341 None] deleting rbd volume volume-4459bb90-d891-42ee-a973-a12bb406a773 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:25:19.294055 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-f45c96ec-2b65-4398-b6fe-ef0346466b9c tempest-TestVolumeBootPattern-747170341 None] Created reservations ['e2e4f6f3-fc7e-43a5-91bb-90ff734f8964', '349948c9-39ee-4408-9918-d32f32fb6417', 'adcf7ec2-9fdc-4603-9203-a7b31b8f53b5', '35cee4a4-52a7-4fea-95ae-d6e525bd8ecb'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:25:19.383278 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-f45c96ec-2b65-4398-b6fe-ef0346466b9c tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:19.452433 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-f45c96ec-2b65-4398-b6fe-ef0346466b9c tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:19.519721 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-f45c96ec-2b65-4398-b6fe-ef0346466b9c tempest-TestVolumeBootPattern-747170341 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:25:19.530129 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-f45c96ec-2b65-4398-b6fe-ef0346466b9c tempest-TestVolumeBootPattern-747170341 None] Deleted volume successfully. Sep 17 21:25:19.547078 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-fedd767e-d2e5-4de9-bf5d-3eb739207e35 req-f45c96ec-2b65-4398-b6fe-ef0346466b9c tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-4459bb90-d891-42ee-a973-a12bb406a773-delete_volume" released by "delete_volume" :: held 0.924s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:25:20.057821 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-88803da5-9c46-46f9-86bf-c7679e2296fb tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-ebc81b3f-de04-4713-a1ca-0d2e7f8eb20f-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:25:20.077629 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-88803da5-9c46-46f9-86bf-c7679e2296fb tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-ebc81b3f-de04-4713-a1ca-0d2e7f8eb20f-delete_snapshot" acquired by "delete_snapshot" :: waited 0.012s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:25:20.081097 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-88803da5-9c46-46f9-86bf-c7679e2296fb tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:25.561603 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:25:25.582262 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Flow 'volume_create_manager' (6b52a472-3709-4c3d-8f2b-76954b575989) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:25:25.588546 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9641258d-7a73-434f-888b-8566604f67d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:25.606302 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9641258d-7a73-434f-888b-8566604f67d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:25:25Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotNegativeTestJSON-Volume-2137694637',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=573e1d1f-a177-495b-8eb0-c759bd47c918,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6db3c501682c4acbb351ad35708c3a4e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:25:26Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:25:26Z,use_quota=True,user_id='7cbd7e6ebfab4ec5a923384e1f357fdb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:25.611791 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7889840a-0057-40dd-829d-c649af5e68a5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:25.616289 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7889840a-0057-40dd-829d-c649af5e68a5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:25.621305 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8e9479b4-f5c0-40be-aea7-cf727a122427) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:25.625350 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (8e9479b4-f5c0-40be-aea7-cf727a122427) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '573e1d1f-a177-495b-8eb0-c759bd47c918', 'volume_name': 'volume-573e1d1f-a177-495b-8eb0-c759bd47c918', 'volume_size': 2}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:25.630227 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (349cd5c3-56eb-4996-9de5-d64047a7e7ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:25.634113 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (349cd5c3-56eb-4996-9de5-d64047a7e7ca) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:25.638799 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7e0f1867-251d-4398-aa38-68d2c4cd194a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:25.639845 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Volume 573e1d1f-a177-495b-8eb0-c759bd47c918: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-573e1d1f-a177-495b-8eb0-c759bd47c918', 'volume_size': 2} Sep 17 21:25:25.640232 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] creating volume 'volume-573e1d1f-a177-495b-8eb0-c759bd47c918' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:25:25.641203 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:25.733650 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:25:25.734201 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:25:25.741496 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7e0f1867-251d-4398-aa38-68d2c4cd194a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-573e1d1f-a177-495b-8eb0-c759bd47c918', 'volume_size': 2}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:25.746872 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (597cb616-0266-4bf8-84b6-1c05068f9a5b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:25.758904 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Volume volume-573e1d1f-a177-495b-8eb0-c759bd47c918 (573e1d1f-a177-495b-8eb0-c759bd47c918): created successfully Sep 17 21:25:25.763970 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (597cb616-0266-4bf8-84b6-1c05068f9a5b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:25.770299 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Flow 'volume_create_manager' (6b52a472-3709-4c3d-8f2b-76954b575989) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:25:25.781090 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c90e7b8c-d646-4080-9c6c-323ecc4b727a tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Created volume successfully. Sep 17 21:25:26.842222 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5efbc262-8735-44f6-8ab3-6717093c3134 tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:27.731365 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5efbc262-8735-44f6-8ab3-6717093c3134 tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Create snapshot completed successfully Sep 17 21:25:28.012074 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-cdb8855f-a4c8-4471-ad3c-52aad128134d tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Acquiring lock "cinder-efb1bdc7-9727-4160-a85a-aaaada477fea-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:25:28.023687 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-cdb8855f-a4c8-4471-ad3c-52aad128134d tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Lock "cinder-efb1bdc7-9727-4160-a85a-aaaada477fea-delete_snapshot" acquired by "delete_snapshot" :: waited 0.011s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:25:28.026426 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-cdb8855f-a4c8-4471-ad3c-52aad128134d tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:28.749216 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-cdb8855f-a4c8-4471-ad3c-52aad128134d tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Created reservations ['2621e40b-3d5c-4e38-8cd9-ff07d21a82ae', 'db4fc015-60ba-4b63-8b69-3e1ba20b70f0', 'da94e487-3c6a-453d-ae05-c4df3710e1c5', '8b70cb91-b3e8-4969-a2e8-58bf112b1796'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:25:28.798535 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-cdb8855f-a4c8-4471-ad3c-52aad128134d tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Delete snapshot completed successfully. Sep 17 21:25:28.812039 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-cdb8855f-a4c8-4471-ad3c-52aad128134d tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Lock "cinder-efb1bdc7-9727-4160-a85a-aaaada477fea-delete_snapshot" released by "delete_snapshot" :: held 0.788s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:25:29.154985 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Acquiring lock "cinder-573e1d1f-a177-495b-8eb0-c759bd47c918-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:25:29.175008 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Lock "cinder-573e1d1f-a177-495b-8eb0-c759bd47c918-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:25:29.233987 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:29.301154 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:25:29.302558 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Volume volume-573e1d1f-a177-495b-8eb0-c759bd47c918 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:25:29.310786 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] deleting rbd volume volume-573e1d1f-a177-495b-8eb0-c759bd47c918 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:25:29.495615 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=62,cinder:INSERT=8,cinder:UPDATE=28,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:29.537284 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Created reservations ['fdfa344a-93a6-430c-8964-23e07784e2ba', '56cbdaac-733c-4b32-9b61-502ca8a17206', 'b5fc8247-970b-456e-a392-96c0a9b8dd5c', '001790c5-3d4e-4aad-8854-957378329cc5'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:25:29.581843 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=53,cinder:INSERT=8,cinder:UPDATE=37,cinder:DELETE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:29.597691 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:29.650548 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:29.715840 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:25:29.720956 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Deleted volume successfully. Sep 17 21:25:29.737818 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-2d075df3-7897-49e5-8e4c-05ec468e6fbe tempest-VolumesSnapshotNegativeTestJSON-1547834551 None] Lock "cinder-573e1d1f-a177-495b-8eb0-c759bd47c918-delete_volume" released by "delete_volume" :: held 0.563s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:25:43.650141 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:25:45.341950 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:25:45.367225 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Flow 'volume_create_manager' (9ffa5ee9-c55e-4016-b384-ff07e090c78c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:25:45.376451 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c38292a4-cb0d-44d0-9b34-ae09af953605) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:45.396663 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c38292a4-cb0d-44d0-9b34-ae09af953605) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:25:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBackupRestore-volume-395351015',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=54b868c4-f2ef-4dd9-a848-b407c7c7ac05,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6e1785b5f4714605b2a35e03c123d54d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:25:45Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:25:45Z,use_quota=True,user_id='23df9a93a08b41f9b3573c1e6a85e60e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:45.405501 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (49050dcd-c156-4267-b15e-a822780d02dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:45.410800 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (49050dcd-c156-4267-b15e-a822780d02dd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:45.421544 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c4b5d710-6f54-43de-a897-c5596c83f445) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:45.560768 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c4b5d710-6f54-43de-a897-c5596c83f445) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '54b868c4-f2ef-4dd9-a848-b407c7c7ac05', 'volume_name': 'volume-54b868c4-f2ef-4dd9-a848-b407c7c7ac05', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:45.566997 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (23d00a49-939c-4016-b9d7-ea3c556598fc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:45.572387 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (23d00a49-939c-4016-b9d7-ea3c556598fc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:45.579030 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1255c7fd-2617-4083-b86f-c86c4db62dfd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:45.580404 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Volume 54b868c4-f2ef-4dd9-a848-b407c7c7ac05: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-54b868c4-f2ef-4dd9-a848-b407c7c7ac05', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 21:25:45.581376 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Cloning 54b868c4-f2ef-4dd9-a848-b407c7c7ac05 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 21:25:45.582541 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:25:45.600060 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.017s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:25:45.620926 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 25, 45, 611581)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:25:45.621786 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:25:45.622335 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 21:25:45.629381 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.029s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:25:45.631132 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 21:25:45.646913 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 25, 45, 639234)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:25:45.647647 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:25:45.648137 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 21:25:45.680083 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:45.801264 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] creating snapshot='volume-54b868c4-f2ef-4dd9-a848-b407c7c7ac05.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:25:46.754603 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-54b868c4-f2ef-4dd9-a848-b407c7c7ac05.clone_snap' to 'volume-54b868c4-f2ef-4dd9-a848-b407c7c7ac05' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:25:46.980823 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:25:46.987843 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:25:46.988195 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:25:47.011515 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:25:47.012132 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Marking volume 54b868c4-f2ef-4dd9-a848-b407c7c7ac05 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:25:47.020970 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to 54b868c4-f2ef-4dd9-a848-b407c7c7ac05. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 21:25:47.021516 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Creating volume glance metadata for volume 54b868c4-f2ef-4dd9-a848-b407c7c7ac05 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 21:25:47.079835 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1255c7fd-2617-4083-b86f-c86c4db62dfd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-54b868c4-f2ef-4dd9-a848-b407c7c7ac05', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:47.086532 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7b161ec0-5090-40a6-a61c-c97987a42875) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:47.103228 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Volume volume-54b868c4-f2ef-4dd9-a848-b407c7c7ac05 (54b868c4-f2ef-4dd9-a848-b407c7c7ac05): created successfully Sep 17 21:25:47.108384 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7b161ec0-5090-40a6-a61c-c97987a42875) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:47.117534 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Flow 'volume_create_manager' (9ffa5ee9-c55e-4016-b384-ff07e090c78c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:25:47.141748 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-de149005-f2c5-4d73-ad62-6aedda63cc25 tempest-TestVolumeBackupRestore-2137501954 None] Created volume successfully. Sep 17 21:25:47.931770 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-efedc231-364f-44a3-8f10-d69c66584dd9 tempest-TestVolumeBackupRestore-2137501954 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-09-17T21:25:47Z,data_timestamp=2022-09-17T21:25:47Z,deleted=False,deleted_at=None,display_description=None,display_name=None,encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=71dc7175-aa1e-474a-98ab-8c27d3df6956,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='6e1785b5f4714605b2a35e03c123d54d',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='23df9a93a08b41f9b3573c1e6a85e60e',volume_id=54b868c4-f2ef-4dd9-a848-b407c7c7ac05) Sep 17 21:25:47.933390 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.backup.rpcapi [None req-efedc231-364f-44a3-8f10-d69c66584dd9 tempest-TestVolumeBackupRestore-2137501954 None] continue_backup in rpcapi backup_id 71dc7175-aa1e-474a-98ab-8c27d3df6956 {{(pid=133409) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} Sep 17 21:25:48.187080 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-efedc231-364f-44a3-8f10-d69c66584dd9 tempest-TestVolumeBackupRestore-2137501954 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:25:49.447468 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-efedc231-364f-44a3-8f10-d69c66584dd9 tempest-TestVolumeBackupRestore-2137501954 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.259s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:25:49.452176 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-efedc231-364f-44a3-8f10-d69c66584dd9 tempest-TestVolumeBackupRestore-2137501954 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-54b868c4-f2ef-4dd9-a848-b407c7c7ac05', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '54b868c4-f2ef-4dd9-a848-b407c7c7ac05', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:25:49.475707 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-efedc231-364f-44a3-8f10-d69c66584dd9 tempest-TestVolumeBackupRestore-2137501954 None] Initialize volume connection completed successfully. Sep 17 21:25:50.518841 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-88803da5-9c46-46f9-86bf-c7679e2296fb tempest-TestVolumeBootPattern-747170341 None] Created reservations ['10d012c3-43f1-48bf-9222-d645b0367c25', 'bc844471-8d65-4237-ac82-a07e4768fc94', 'b0808285-ece8-44fa-825f-19a035500c81', 'c6620903-1734-4dea-acbb-ca5477946097'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:25:50.646275 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-88803da5-9c46-46f9-86bf-c7679e2296fb tempest-TestVolumeBootPattern-747170341 None] Delete snapshot completed successfully. Sep 17 21:25:50.653000 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-88803da5-9c46-46f9-86bf-c7679e2296fb tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-ebc81b3f-de04-4713-a1ca-0d2e7f8eb20f-delete_snapshot" released by "delete_snapshot" :: held 30.583s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:25:55.665987 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-25aaef97-2cd8-4011-aa99-2137678be5b6 tempest-TestVolumeBootPattern-747170341 None] Terminate volume connection completed successfully. Sep 17 21:25:55.676118 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-25aaef97-2cd8-4011-aa99-2137678be5b6 tempest-TestVolumeBootPattern-747170341 None] Deleting attachment 80f26435-5f31-4392-bf40-d91ad1649289. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:25:55.904689 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-181d1f0d-1ec7-49df-8b5b-e3c264fefd29-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:25:55.916664 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-181d1f0d-1ec7-49df-8b5b-e3c264fefd29-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:25:55.982997 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:56.118677 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:25:56.130822 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] deleting rbd volume volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:25:56.585071 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:25:56.624056 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] deleting parent snapshot volume-181d1f0d-1ec7-49df-8b5b-e3c264fefd29.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:25:57.861899 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-efedc231-364f-44a3-8f10-d69c66584dd9 tempest-TestVolumeBackupRestore-2137501954 None] Terminate volume connection completed successfully. Sep 17 21:25:57.885817 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-efedc231-364f-44a3-8f10-d69c66584dd9 tempest-TestVolumeBackupRestore-2137501954 None] Remove volume export completed successfully. Sep 17 21:25:57.924024 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] Created reservations ['545634a6-de8c-46d2-bac6-2f4d0247fde7', 'e73a5428-1fc4-4317-a766-33c17d670c65', 'a8137f11-602a-4ed3-a682-04fbea7367d0', 'e2952533-77e0-463b-9077-a4f0fc1adeb4'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:25:58.006647 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:58.056052 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:58.098493 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:25:58.103106 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] Deleted volume successfully. Sep 17 21:25:58.122584 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-5b8e37a3-4dcf-4597-ad94-46704d178e59 req-c1324039-5297-4f44-9f66-9794c990430d tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-181d1f0d-1ec7-49df-8b5b-e3c264fefd29-delete_volume" released by "delete_volume" :: held 2.206s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:25:58.836317 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:25:58.857419 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Flow 'volume_create_manager' (e9e600f3-28c8-4080-b68c-0628b2fee461) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:25:58.864869 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (87f9035e-d6c6-4f48-a2a7-fbc3f6d526bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:58.882599 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (87f9035e-d6c6-4f48-a2a7-fbc3f6d526bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:25:59Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_71dc7175-aa1e-474a-98ab-8c27d3df6956',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bdf805c4-767e-4378-a2eb-85d92fa21105,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6e1785b5f4714605b2a35e03c123d54d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:25:59Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:25:59Z,use_quota=True,user_id='23df9a93a08b41f9b3573c1e6a85e60e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:58.891080 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (52bd9c09-c5e5-428f-b84e-c50787342555) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:58.895386 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (52bd9c09-c5e5-428f-b84e-c50787342555) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:58.901314 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dca4c009-1bf1-480a-9534-c095c5eef348) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:58.907602 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dca4c009-1bf1-480a-9534-c095c5eef348) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'bdf805c4-767e-4378-a2eb-85d92fa21105', 'volume_name': 'volume-bdf805c4-767e-4378-a2eb-85d92fa21105', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:58.912551 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac660df3-42da-4618-829d-d42aad5473d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:58.917233 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac660df3-42da-4618-829d-d42aad5473d7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:58.921896 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1261172c-67c6-4d16-9055-ca927f3d7430) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:58.922998 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Volume bdf805c4-767e-4378-a2eb-85d92fa21105: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-bdf805c4-767e-4378-a2eb-85d92fa21105', 'volume_size': 1} Sep 17 21:25:58.923460 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] creating volume 'volume-bdf805c4-767e-4378-a2eb-85d92fa21105' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:25:58.924386 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:25:59.056572 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:25:59.057194 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:25:59.061076 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1261172c-67c6-4d16-9055-ca927f3d7430) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bdf805c4-767e-4378-a2eb-85d92fa21105', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:59.068223 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (52df77de-8080-445f-b5ee-033b09a690fb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:25:59.083035 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Volume volume-bdf805c4-767e-4378-a2eb-85d92fa21105 (bdf805c4-767e-4378-a2eb-85d92fa21105): created successfully Sep 17 21:25:59.087730 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (52df77de-8080-445f-b5ee-033b09a690fb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:25:59.093928 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Flow 'volume_create_manager' (e9e600f3-28c8-4080-b68c-0628b2fee461) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:25:59.103675 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Created volume successfully. Sep 17 21:25:59.947773 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:26:00.902070 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.954s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:26:00.902855 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-bdf805c4-767e-4378-a2eb-85d92fa21105', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'bdf805c4-767e-4378-a2eb-85d92fa21105', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:26:00.911751 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=57,cinder:INSERT=13,cinder:UPDATE=26,cinder:DELETE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:00.921255 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Initialize volume connection completed successfully. Sep 17 21:26:01.033848 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:26:01.034925 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:26:01.094445 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:26:01.160081 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:26:02.412292 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:26:02.451448 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (2c73741f-a593-41ac-a8cf-f42111c2f374) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:26:02.467615 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0bb7232a-9531-4e58-a4ed-bc50afaeb4f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:26:02.489849 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0bb7232a-9531-4e58-a4ed-bc50afaeb4f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:26:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-111176100',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=43df5366-32dd-4c44-ad36-15410ac6ac72,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b99d2a0b4d7e40a38e682fc147e968af',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:26:02Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:26:02Z,use_quota=True,user_id='342e60103c9b45bcb348b6f8bcd686ca',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:26:02.497065 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7348b79c-6a1d-4e79-bcf7-d9b040864ad8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:26:02.504207 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7348b79c-6a1d-4e79-bcf7-d9b040864ad8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:26:02.509888 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (43b837b6-e25d-48f4-8c49-fc96a78bc484) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:26:02.651727 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (43b837b6-e25d-48f4-8c49-fc96a78bc484) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '43df5366-32dd-4c44-ad36-15410ac6ac72', 'volume_name': 'volume-43df5366-32dd-4c44-ad36-15410ac6ac72', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:26:02.665555 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7365f2dc-69fb-4366-a16a-8d62a8774510) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:26:02.680672 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7365f2dc-69fb-4366-a16a-8d62a8774510) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:26:02.692461 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bc4d49f3-09b5-4a06-a129-174477245454) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:26:02.700102 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Volume 43df5366-32dd-4c44-ad36-15410ac6ac72: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-43df5366-32dd-4c44-ad36-15410ac6ac72', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 21:26:02.701162 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Cloning 43df5366-32dd-4c44-ad36-15410ac6ac72 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 21:26:02.702496 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:26:02.720647 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.018s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:26:02.740317 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 26, 2, 731743)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:26:02.741308 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:26:02.742005 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 21:26:02.756723 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.036s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:26:02.759498 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 21:26:02.779701 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 21, 26, 2, 771449)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 21:26:02.782130 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 21:26:02.782700 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 21:26:02.811453 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:26:02.946207 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] creating snapshot='volume-43df5366-32dd-4c44-ad36-15410ac6ac72.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 21:26:04.068647 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-43df5366-32dd-4c44-ad36-15410ac6ac72.clone_snap' to 'volume-43df5366-32dd-4c44-ad36-15410ac6ac72' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 21:26:04.522155 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:26:04.523757 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:26:04.524474 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:26:04.632210 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 21:26:04.633312 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Marking volume 43df5366-32dd-4c44-ad36-15410ac6ac72 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:26:04.650592 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to 43df5366-32dd-4c44-ad36-15410ac6ac72. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 21:26:04.651313 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Creating volume glance metadata for volume 43df5366-32dd-4c44-ad36-15410ac6ac72 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 21:26:04.736019 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=64,cinder:UPDATE=23,cinder:INSERT=12,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:04.819257 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bc4d49f3-09b5-4a06-a129-174477245454) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-43df5366-32dd-4c44-ad36-15410ac6ac72', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:26:04.824830 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (28af92ef-8d10-44de-b4bb-e80864889331) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:26:04.856386 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Volume volume-43df5366-32dd-4c44-ad36-15410ac6ac72 (43df5366-32dd-4c44-ad36-15410ac6ac72): created successfully Sep 17 21:26:04.860845 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (28af92ef-8d10-44de-b4bb-e80864889331) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:26:04.867721 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (2c73741f-a593-41ac-a8cf-f42111c2f374) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:26:04.884675 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-a45528d5-0b1a-4134-8cce-98ab857a28b6 tempest-TestVolumeBootPattern-747170341 None] Created volume successfully. Sep 17 21:26:06.304451 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Terminate volume connection completed successfully. Sep 17 21:26:06.335073 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-28adacb0-dc5d-4257-8323-4512b51aba8b tempest-TestVolumeBackupRestore-2137501954 None] Remove volume export completed successfully. Sep 17 21:26:08.164992 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-b6dbf603-b60b-4575-86ad-d339834958c8 req-898e39f3-8964-45fd-bb2d-eb2220385c64 tempest-TestVolumeBootPattern-747170341 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:26:09.123076 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-b6dbf603-b60b-4575-86ad-d339834958c8 req-898e39f3-8964-45fd-bb2d-eb2220385c64 tempest-TestVolumeBootPattern-747170341 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.958s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:26:09.124377 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-b6dbf603-b60b-4575-86ad-d339834958c8 req-898e39f3-8964-45fd-bb2d-eb2220385c64 tempest-TestVolumeBootPattern-747170341 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-43df5366-32dd-4c44-ad36-15410ac6ac72', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '43df5366-32dd-4c44-ad36-15410ac6ac72', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:26:09.152492 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-b6dbf603-b60b-4575-86ad-d339834958c8 req-898e39f3-8964-45fd-bb2d-eb2220385c64 tempest-TestVolumeBootPattern-747170341 None] Connection info returned from driver {'name': 'volumes/volume-43df5366-32dd-4c44-ad36-15410ac6ac72', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '43df5366-32dd-4c44-ad36-15410ac6ac72', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '720964ab-f928-4611-96d2-2f566aef449f'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:26:09.222053 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-b6dbf603-b60b-4575-86ad-d339834958c8 req-898e39f3-8964-45fd-bb2d-eb2220385c64 tempest-TestVolumeBootPattern-747170341 None] attachment_update completed successfully. Sep 17 21:26:11.573586 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-4f5b5fcc-5605-43c8-8f02-53038a09729b req-390c084f-aa83-4ca3-9b1a-dcd4b118f4bc tempest-TestVolumeBackupRestore-2137501954 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:26:12.416277 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-4f5b5fcc-5605-43c8-8f02-53038a09729b req-390c084f-aa83-4ca3-9b1a-dcd4b118f4bc tempest-TestVolumeBackupRestore-2137501954 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.842s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:26:12.417443 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-4f5b5fcc-5605-43c8-8f02-53038a09729b req-390c084f-aa83-4ca3-9b1a-dcd4b118f4bc tempest-TestVolumeBackupRestore-2137501954 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-bdf805c4-767e-4378-a2eb-85d92fa21105', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'bdf805c4-767e-4378-a2eb-85d92fa21105', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:26:12.446118 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-4f5b5fcc-5605-43c8-8f02-53038a09729b req-390c084f-aa83-4ca3-9b1a-dcd4b118f4bc tempest-TestVolumeBackupRestore-2137501954 None] Connection info returned from driver {'name': 'volumes/volume-bdf805c4-767e-4378-a2eb-85d92fa21105', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'bdf805c4-767e-4378-a2eb-85d92fa21105', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'dc1be6ea-dcb6-46a7-bee7-ff29529e64e5'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:26:12.506060 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-4f5b5fcc-5605-43c8-8f02-53038a09729b req-390c084f-aa83-4ca3-9b1a-dcd4b118f4bc tempest-TestVolumeBackupRestore-2137501954 None] attachment_update completed successfully. Sep 17 21:26:22.481257 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=43,cinder:UPDATE=11,cinder:INSERT=9,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:44.081229 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=36,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:26:44.083013 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:27:01.032857 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:27:01.034235 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:27:01.068967 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:27:01.095840 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:27:10.366922 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-6ced1e0c-444e-43eb-a3cf-913953d61663 req-d758de2d-51e0-4825-937f-780fe50854ee tempest-TestVolumeBackupRestore-2137501954 None] Terminate volume connection completed successfully. Sep 17 21:27:10.367620 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-6ced1e0c-444e-43eb-a3cf-913953d61663 req-d758de2d-51e0-4825-937f-780fe50854ee tempest-TestVolumeBackupRestore-2137501954 None] Deleting attachment dc1be6ea-dcb6-46a7-bee7-ff29529e64e5. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:27:12.131599 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-92c65610-c23a-43e9-b8a4-c6ab558257a6 tempest-TestVolumeBackupRestore-2137501954 None] Acquiring lock "cinder-bdf805c4-767e-4378-a2eb-85d92fa21105-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:27:12.141909 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-92c65610-c23a-43e9-b8a4-c6ab558257a6 tempest-TestVolumeBackupRestore-2137501954 None] Lock "cinder-bdf805c4-767e-4378-a2eb-85d92fa21105-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:27:12.179936 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-92c65610-c23a-43e9-b8a4-c6ab558257a6 tempest-TestVolumeBackupRestore-2137501954 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:27:12.829899 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-92c65610-c23a-43e9-b8a4-c6ab558257a6 tempest-TestVolumeBackupRestore-2137501954 None] Volume volume-bdf805c4-767e-4378-a2eb-85d92fa21105 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:27:12.849986 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-92c65610-c23a-43e9-b8a4-c6ab558257a6 tempest-TestVolumeBackupRestore-2137501954 None] deleting rbd volume volume-bdf805c4-767e-4378-a2eb-85d92fa21105 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:27:12.855104 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] Acquiring lock "cinder-54b868c4-f2ef-4dd9-a848-b407c7c7ac05-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:27:12.867032 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] Lock "cinder-54b868c4-f2ef-4dd9-a848-b407c7c7ac05-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:27:12.903311 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:27:13.848283 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] deleting rbd volume volume-54b868c4-f2ef-4dd9-a848-b407c7c7ac05 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:27:13.899431 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-92c65610-c23a-43e9-b8a4-c6ab558257a6 tempest-TestVolumeBackupRestore-2137501954 None] Created reservations ['616a2934-81b9-467e-b9e7-e24dabc44cd2', 'e497ab44-fef1-425e-a9e0-21792ac38fc9', '97f0acea-7214-44bc-b43b-9181555d48e7', '150e3ba6-c4a4-48ed-930a-b51fa64b3349'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:27:13.959660 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-92c65610-c23a-43e9-b8a4-c6ab558257a6 tempest-TestVolumeBackupRestore-2137501954 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:27:13.990498 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-92c65610-c23a-43e9-b8a4-c6ab558257a6 tempest-TestVolumeBackupRestore-2137501954 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:27:14.019092 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-92c65610-c23a-43e9-b8a4-c6ab558257a6 tempest-TestVolumeBackupRestore-2137501954 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:27:14.028976 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-92c65610-c23a-43e9-b8a4-c6ab558257a6 tempest-TestVolumeBackupRestore-2137501954 None] Deleted volume successfully. Sep 17 21:27:14.032083 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:27:14.062074 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] deleting parent snapshot volume-54b868c4-f2ef-4dd9-a848-b407c7c7ac05.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:27:14.867615 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=22,cinder:UPDATE=11,cinder:INSERT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:27:14.868263 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=23,cinder:UPDATE=9,cinder:INSERT=3,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:27:14.889939 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-92c65610-c23a-43e9-b8a4-c6ab558257a6 tempest-TestVolumeBackupRestore-2137501954 None] Lock "cinder-bdf805c4-767e-4378-a2eb-85d92fa21105-delete_volume" released by "delete_volume" :: held 2.748s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:27:14.915047 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] Created reservations ['a977970f-666d-44d4-8c14-e8153fbc0563', '45c30579-7bc7-4971-9249-e80b91589653', '4929366b-f1ea-48f1-83d0-b18d42853093', '64ade42c-a90d-44fd-a9c0-7434e3cb05f5'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:27:14.957279 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:27:14.982422 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:27:15.008300 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:27:15.012388 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] Deleted volume successfully. Sep 17 21:27:15.025244 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-315df4f8-f78b-4e21-aaba-bcedaa63656a tempest-TestVolumeBackupRestore-2137501954 None] Lock "cinder-54b868c4-f2ef-4dd9-a848-b407c7c7ac05-delete_volume" released by "delete_volume" :: held 2.158s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:27:54.866393 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=20,cinder:UPDATE=9,cinder:INSERT=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:27:54.868447 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=9,cinder:INSERT=1,cinder:UPDATE=10,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:28:01.032763 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:28:01.033856 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:28:01.068384 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:28:01.092939 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:28:14.872743 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:28:28.790457 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:28:28.811770 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-cbbf1726-22a4-44fa-9d29-6425b5858b78 req-490ba748-213b-4a57-b8f2-a7bf171f11dd tempest-TestVolumeBootPattern-747170341 None] Terminate volume connection completed successfully. Sep 17 21:28:28.812195 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-cbbf1726-22a4-44fa-9d29-6425b5858b78 req-490ba748-213b-4a57-b8f2-a7bf171f11dd tempest-TestVolumeBootPattern-747170341 None] Deleting attachment 720964ab-f928-4611-96d2-2f566aef449f. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:28:33.256664 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-9d017a1d-a633-4368-8395-e9be239b80f3 req-afb95d50-d928-4b12-a8a7-e028c114d394 tempest-TestVolumeBootPattern-747170341 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:28:34.056508 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-9d017a1d-a633-4368-8395-e9be239b80f3 req-afb95d50-d928-4b12-a8a7-e028c114d394 tempest-TestVolumeBootPattern-747170341 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.799s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:28:34.057558 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-9d017a1d-a633-4368-8395-e9be239b80f3 req-afb95d50-d928-4b12-a8a7-e028c114d394 tempest-TestVolumeBootPattern-747170341 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-43df5366-32dd-4c44-ad36-15410ac6ac72', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '43df5366-32dd-4c44-ad36-15410ac6ac72', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:28:34.083032 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-9d017a1d-a633-4368-8395-e9be239b80f3 req-afb95d50-d928-4b12-a8a7-e028c114d394 tempest-TestVolumeBootPattern-747170341 None] Connection info returned from driver {'name': 'volumes/volume-43df5366-32dd-4c44-ad36-15410ac6ac72', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '43df5366-32dd-4c44-ad36-15410ac6ac72', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '648925ca-c06f-45e7-8ec8-aeb5c1d6692e'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:28:34.133226 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-9d017a1d-a633-4368-8395-e9be239b80f3 req-afb95d50-d928-4b12-a8a7-e028c114d394 tempest-TestVolumeBootPattern-747170341 None] attachment_update completed successfully. Sep 17 21:28:44.132020 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=17 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:28:54.886380 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=13,cinder:UPDATE=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:29:01.028436 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:29:01.029258 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:29:01.055903 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:29:01.080392 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:29:14.885224 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:29:34.890318 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:29:54.879840 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:30:01.026786 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:30:01.028304 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:30:01.053902 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:30:01.081604 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:30:04.889934 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:30:24.926386 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:30:24.956473 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1d383179-2319-4060-b936-4c1fe2bdd390 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:30:25.294779 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-1d383179-2319-4060-b936-4c1fe2bdd390 tempest-TestVolumeBootPattern-747170341 None] Create snapshot completed successfully Sep 17 21:30:26.404727 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:30:26.432816 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (63b20b49-396a-4c33-a48b-d6ce05a5d2ac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:30:26.439215 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e14503a3-700f-4e6f-bf31-ffdbe215c72c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:30:26.455499 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e14503a3-700f-4e6f-bf31-ffdbe215c72c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:30:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-202426277',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b99d2a0b4d7e40a38e682fc147e968af',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:30:26Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=c4cc986b-79ab-4375-84af-0426fa5c5ae3,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:30:26Z,use_quota=True,user_id='342e60103c9b45bcb348b6f8bcd686ca',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:30:26.460512 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (db35cd95-0c86-4958-9174-80dd07c469f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:30:26.464419 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (db35cd95-0c86-4958-9174-80dd07c469f9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:30:26.469341 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (527c25f3-406a-43ce-aaba-ff20d1346e12) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:30:26.473454 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (527c25f3-406a-43ce-aaba-ff20d1346e12) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac', 'volume_name': 'volume-6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac', 'volume_size': 1, 'snapshot_id': 'c4cc986b-79ab-4375-84af-0426fa5c5ae3'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:30:26.478081 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (81690220-31d4-46c7-8442-abf081179035) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:30:26.481782 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (81690220-31d4-46c7-8442-abf081179035) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:30:26.486974 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0a5bd9c7-040d-4c15-99ca-b27e178bc753) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:30:26.488100 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Volume 6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac', 'volume_size': 1, 'snapshot_id': 'c4cc986b-79ab-4375-84af-0426fa5c5ae3'} Sep 17 21:30:26.511120 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] cloning volumes/volume-43df5366-32dd-4c44-ad36-15410ac6ac72@snapshot-c4cc986b-79ab-4375-84af-0426fa5c5ae3 to volume-6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac {{(pid=133409) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1184}} Sep 17 21:30:26.511751 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:30:26.572436 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:30:26.695804 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:30:26.696388 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:30:26.718608 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Marking volume 6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 21:30:26.728050 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Copying metadata from snapshot c4cc986b-79ab-4375-84af-0426fa5c5ae3 to 6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:430}} Sep 17 21:30:26.762665 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0a5bd9c7-040d-4c15-99ca-b27e178bc753) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac', 'volume_size': 1, 'snapshot_id': 'c4cc986b-79ab-4375-84af-0426fa5c5ae3'}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:30:26.767131 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fa847820-40c2-4998-b544-e54c5ce1f8ce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:30:26.778709 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Volume volume-6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac (6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac): created successfully Sep 17 21:30:26.782820 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fa847820-40c2-4998-b544-e54c5ce1f8ce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:30:26.787406 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Flow 'volume_create_manager' (63b20b49-396a-4c33-a48b-d6ce05a5d2ac) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:30:26.800136 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5ccbf076-25a3-453c-9794-bc9895b080d7 tempest-TestVolumeBootPattern-747170341 None] Created volume successfully. Sep 17 21:30:29.703759 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-421444f3-24c4-4446-9e30-6fca79f31c0b req-70f10479-b275-44b0-9191-49d59bb28d67 tempest-TestVolumeBootPattern-747170341 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:30:30.513044 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-421444f3-24c4-4446-9e30-6fca79f31c0b req-70f10479-b275-44b0-9191-49d59bb28d67 tempest-TestVolumeBootPattern-747170341 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.809s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:30:30.514030 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-421444f3-24c4-4446-9e30-6fca79f31c0b req-70f10479-b275-44b0-9191-49d59bb28d67 tempest-TestVolumeBootPattern-747170341 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:30:30.540105 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-421444f3-24c4-4446-9e30-6fca79f31c0b req-70f10479-b275-44b0-9191-49d59bb28d67 tempest-TestVolumeBootPattern-747170341 None] Connection info returned from driver {'name': 'volumes/volume-6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '51ae5945-63cc-43f8-8a0d-90200c5ae6a2'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:30:30.605498 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-421444f3-24c4-4446-9e30-6fca79f31c0b req-70f10479-b275-44b0-9191-49d59bb28d67 tempest-TestVolumeBootPattern-747170341 None] attachment_update completed successfully. Sep 17 21:30:44.876290 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=28,cinder:INSERT=14,cinder:UPDATE=2,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:30:44.877018 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=39,cinder:UPDATE=9,cinder:INSERT=10,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:31:01.027777 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:31:01.028730 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:31:01.063957 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:31:01.089108 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:31:04.882709 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:31:24.876874 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:31:44.878053 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:31:44.880697 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:32:01.024040 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:32:01.024678 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:32:01.054481 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:32:01.085075 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:32:14.878175 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:32:24.876336 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:32:27.840866 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-f210a889-8148-4fc0-92ac-fbf9fc7848a0 req-25b4b7e5-dfa5-4efd-a5d9-3f02d98d24fc tempest-TestVolumeBootPattern-747170341 None] Terminate volume connection completed successfully. Sep 17 21:32:27.841567 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-f210a889-8148-4fc0-92ac-fbf9fc7848a0 req-25b4b7e5-dfa5-4efd-a5d9-3f02d98d24fc tempest-TestVolumeBootPattern-747170341 None] Deleting attachment 51ae5945-63cc-43f8-8a0d-90200c5ae6a2. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:32:29.329604 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-bc4fcd58-98ea-422c-9867-3c51edd85f0c tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:32:29.338757 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-bc4fcd58-98ea-422c-9867-3c51edd85f0c tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac-delete_volume" acquired by "delete_volume" :: waited 0.010s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:32:29.372867 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-bc4fcd58-98ea-422c-9867-3c51edd85f0c tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:32:29.455278 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-bc4fcd58-98ea-422c-9867-3c51edd85f0c tempest-TestVolumeBootPattern-747170341 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:32:29.464461 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-bc4fcd58-98ea-422c-9867-3c51edd85f0c tempest-TestVolumeBootPattern-747170341 None] deleting rbd volume volume-6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:32:29.700149 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-bc4fcd58-98ea-422c-9867-3c51edd85f0c tempest-TestVolumeBootPattern-747170341 None] Created reservations ['2d52a5d8-8277-4358-a29b-c2c20db8d862', 'fb108170-871e-4964-acfb-5cdab4b24f4b', '063a4577-0851-4250-bc50-e328809d18d3', '35bbf756-8055-4878-992e-26927474af79'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:32:29.750958 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-bc4fcd58-98ea-422c-9867-3c51edd85f0c tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:32:29.775696 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-bc4fcd58-98ea-422c-9867-3c51edd85f0c tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:32:29.801079 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-bc4fcd58-98ea-422c-9867-3c51edd85f0c tempest-TestVolumeBootPattern-747170341 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:32:29.805632 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-bc4fcd58-98ea-422c-9867-3c51edd85f0c tempest-TestVolumeBootPattern-747170341 None] Deleted volume successfully. Sep 17 21:32:29.820725 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-bc4fcd58-98ea-422c-9867-3c51edd85f0c tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-6a0cfe92-0d9c-46d1-91d9-ddfcb8b8ceac-delete_volume" released by "delete_volume" :: held 0.482s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:32:30.478322 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-57034a8f-326b-4fba-8283-7c8aa4aa9d81 tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-c4cc986b-79ab-4375-84af-0426fa5c5ae3-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:32:30.487922 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-57034a8f-326b-4fba-8283-7c8aa4aa9d81 tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-c4cc986b-79ab-4375-84af-0426fa5c5ae3-delete_snapshot" acquired by "delete_snapshot" :: waited 0.010s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:32:30.489653 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-57034a8f-326b-4fba-8283-7c8aa4aa9d81 tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:32:54.873331 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=19,cinder:INSERT=2,cinder:UPDATE=10 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:32:54.874161 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=18,cinder:UPDATE=9,cinder:INSERT=2,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:00.667684 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-57034a8f-326b-4fba-8283-7c8aa4aa9d81 tempest-TestVolumeBootPattern-747170341 None] Created reservations ['306bf39a-fc55-4f05-ac3b-84c2f136aabd', 'c7f139b0-9eda-4385-b099-aca6b592ed9d', '0eff1cdd-dd2f-42c3-b1c9-f0d7f3d77277', '026652a1-c878-4ba9-a266-a8d3afe27a08'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:33:00.711937 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-57034a8f-326b-4fba-8283-7c8aa4aa9d81 tempest-TestVolumeBootPattern-747170341 None] Delete snapshot completed successfully. Sep 17 21:33:00.716771 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-57034a8f-326b-4fba-8283-7c8aa4aa9d81 tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-c4cc986b-79ab-4375-84af-0426fa5c5ae3-delete_snapshot" released by "delete_snapshot" :: held 30.229s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:33:01.021264 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:33:01.022222 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:33:01.048988 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:33:01.076799 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:33:04.934886 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-d1c69cbe-f4a9-46e2-be87-20115eaca909 req-7420e92f-60e7-4e82-bba7-da22c84a7457 tempest-TestVolumeBootPattern-747170341 None] Terminate volume connection completed successfully. Sep 17 21:33:04.935596 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-d1c69cbe-f4a9-46e2-be87-20115eaca909 req-7420e92f-60e7-4e82-bba7-da22c84a7457 tempest-TestVolumeBootPattern-747170341 None] Deleting attachment 648925ca-c06f-45e7-8ec8-aeb5c1d6692e. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:33:07.055958 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] Acquiring lock "cinder-43df5366-32dd-4c44-ad36-15410ac6ac72-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:33:07.065443 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-43df5366-32dd-4c44-ad36-15410ac6ac72-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:33:07.103305 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:33:07.173206 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:33:07.179486 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] deleting rbd volume volume-43df5366-32dd-4c44-ad36-15410ac6ac72 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:33:07.403541 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 21:33:07.434730 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] deleting parent snapshot volume-43df5366-32dd-4c44-ad36-15410ac6ac72.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 21:33:07.719979 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] Created reservations ['53771502-39dc-4731-a6b1-33ef93e74f4d', 'b5bc652a-458e-45d8-9b4f-ebe5f249de22', '6328d469-be3c-435d-88ba-03dfe5789361', '4f525c9e-a09a-434c-963d-61ee5a77e51f'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:33:07.766374 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:33:07.794055 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:33:07.818797 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:33:07.823065 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] Deleted volume successfully. Sep 17 21:33:07.838766 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-5b9228c3-3107-4e66-9f46-d17d6ddb99ec tempest-TestVolumeBootPattern-747170341 None] Lock "cinder-43df5366-32dd-4c44-ad36-15410ac6ac72-delete_volume" released by "delete_volume" :: held 0.773s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:33:34.881137 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=26,cinder:UPDATE=16,cinder:INSERT=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:34.882322 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=32,cinder:INSERT=3,cinder:UPDATE=15,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:33:54.880788 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:34:01.022027 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:34:01.022695 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:34:01.049332 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:34:01.080278 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:34:14.886244 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:34:34.886880 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:34:54.892197 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:35:01.025137 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:35:01.025851 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:35:01.053732 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:35:01.097290 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:35:14.890338 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=8,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:35:34.895880 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:35:44.902246 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:36:01.024959 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:36:01.025731 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:36:01.062516 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:36:01.089849 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:36:04.905315 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:36:14.898097 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:36:34.906086 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:36:54.906814 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:37:01.025925 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:37:01.026965 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:37:01.062981 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:37:01.089849 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:37:14.904365 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:37:34.903495 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:37:54.902266 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:38:14.896092 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:38:34.897683 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:38:34.900643 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:39:01.026066 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:39:01.026853 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:39:01.051439 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:39:01.084068 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:39:04.906835 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:39:14.900367 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:39:44.903624 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:40:01.030029 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:40:01.031237 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:40:01.068396 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:40:01.092463 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:40:04.896936 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:40:24.898864 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:40:34.910130 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:40:54.907431 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:40:54.919851 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:01.028844 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:41:01.029903 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:41:01.057787 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:41:01.081530 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:41:03.454858 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:41:03.474199 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Flow 'volume_create_manager' (1cbdf523-baf1-4914-9456-3651f66cb923) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:41:03.482009 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (405b2532-37f9-400c-85e5-02564fb88464) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:41:03.498037 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (405b2532-37f9-400c-85e5-02564fb88464) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:41:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerActionsTestJSON-volume-486511017',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ad243abc13a14aa08c717d4105e24f4f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:41:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:41:03Z,use_quota=True,user_id='d20f8598dfec4de7b496a64e023e7a15',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:41:03.503091 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (55993865-32a2-46d1-ad8b-62bf82f25d19) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:41:03.507070 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (55993865-32a2-46d1-ad8b-62bf82f25d19) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:41:03.511929 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (316b6be4-dcef-4ab6-9c55-67ac0ffffdef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:41:03.515830 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (316b6be4-dcef-4ab6-9c55-67ac0ffffdef) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'volume_name': 'volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:41:03.520467 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (360506b5-a85a-4884-856d-8a92eb5eeee7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:41:03.524201 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (360506b5-a85a-4884-856d-8a92eb5eeee7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:41:03.528686 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (957e5e9a-677d-4dce-b4a7-5c5bfa9f9dcc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:41:03.529731 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Volume 1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'volume_size': 1} Sep 17 21:41:03.530117 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] creating volume 'volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:41:03.530872 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:41:03.581007 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:41:03.581367 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:41:03.585006 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (957e5e9a-677d-4dce-b4a7-5c5bfa9f9dcc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:41:03.589634 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3269d9d6-c547-4cd8-ba37-ee77277d488a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:41:03.602365 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Volume volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b (1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b): created successfully Sep 17 21:41:03.605930 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3269d9d6-c547-4cd8-ba37-ee77277d488a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:41:03.611319 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Flow 'volume_create_manager' (1cbdf523-baf1-4914-9456-3651f66cb923) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:41:03.620902 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-1a1c6c37-1291-4396-95fc-91c47996871e tempest-ServerActionsTestJSON-226995111 None] Created volume successfully. Sep 17 21:41:05.097686 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-b517934e-fb96-4a9d-b4d7-f500e51a8487 req-755c6922-32e8-4968-8f75-c3ac25023e9b tempest-ServerActionsTestJSON-226995111 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:41:05.848835 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-b517934e-fb96-4a9d-b4d7-f500e51a8487 req-755c6922-32e8-4968-8f75-c3ac25023e9b tempest-ServerActionsTestJSON-226995111 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.751s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:41:05.849704 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-b517934e-fb96-4a9d-b4d7-f500e51a8487 req-755c6922-32e8-4968-8f75-c3ac25023e9b tempest-ServerActionsTestJSON-226995111 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:41:05.876283 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-b517934e-fb96-4a9d-b4d7-f500e51a8487 req-755c6922-32e8-4968-8f75-c3ac25023e9b tempest-ServerActionsTestJSON-226995111 None] Connection info returned from driver {'name': 'volumes/volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '5cad5621-c8f5-491d-98b2-d3940cd39f5c'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:41:05.925722 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-b517934e-fb96-4a9d-b4d7-f500e51a8487 req-755c6922-32e8-4968-8f75-c3ac25023e9b tempest-ServerActionsTestJSON-226995111 None] attachment_update completed successfully. Sep 17 21:41:22.633494 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-036f32d1-c1ed-4f73-940f-ffbd2522746f req-369a0a61-9794-4dc0-88f0-d56e77abe6eb tempest-ServerActionsTestJSON-226995111 None] Terminate volume connection completed successfully. Sep 17 21:41:22.634173 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-036f32d1-c1ed-4f73-940f-ffbd2522746f req-369a0a61-9794-4dc0-88f0-d56e77abe6eb tempest-ServerActionsTestJSON-226995111 None] Deleting attachment 5cad5621-c8f5-491d-98b2-d3940cd39f5c. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:41:24.719598 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-036f32d1-c1ed-4f73-940f-ffbd2522746f req-4d26509b-b1f4-4720-91ea-5153f533bf8c tempest-ServerActionsTestJSON-226995111 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:41:25.461905 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-036f32d1-c1ed-4f73-940f-ffbd2522746f req-4d26509b-b1f4-4720-91ea-5153f533bf8c tempest-ServerActionsTestJSON-226995111 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.742s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:41:25.462667 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-036f32d1-c1ed-4f73-940f-ffbd2522746f req-4d26509b-b1f4-4720-91ea-5153f533bf8c tempest-ServerActionsTestJSON-226995111 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:41:25.465563 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=25,cinder:UPDATE=3,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:25.467728 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=24,cinder:UPDATE=6 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:41:25.491535 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-036f32d1-c1ed-4f73-940f-ffbd2522746f req-4d26509b-b1f4-4720-91ea-5153f533bf8c tempest-ServerActionsTestJSON-226995111 None] Connection info returned from driver {'name': 'volumes/volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '9d5afdd6-1c34-4f7d-9ede-2e186692f084'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:41:25.541554 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-036f32d1-c1ed-4f73-940f-ffbd2522746f req-4d26509b-b1f4-4720-91ea-5153f533bf8c tempest-ServerActionsTestJSON-226995111 None] attachment_update completed successfully. Sep 17 21:41:32.998450 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-7e293e16-c440-453a-a851-dd561ed8ca58 req-08b3dff3-5df0-456b-b99d-44f96dc48f52 tempest-ServerActionsTestJSON-226995111 None] Terminate volume connection completed successfully. Sep 17 21:41:32.999002 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-7e293e16-c440-453a-a851-dd561ed8ca58 req-08b3dff3-5df0-456b-b99d-44f96dc48f52 tempest-ServerActionsTestJSON-226995111 None] Deleting attachment 9d5afdd6-1c34-4f7d-9ede-2e186692f084. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:41:35.427348 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-7e293e16-c440-453a-a851-dd561ed8ca58 req-0cb95b6a-4bf1-4647-9c86-8b696aaface6 tempest-ServerActionsTestJSON-226995111 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:41:36.176254 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-7e293e16-c440-453a-a851-dd561ed8ca58 req-0cb95b6a-4bf1-4647-9c86-8b696aaface6 tempest-ServerActionsTestJSON-226995111 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.748s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:41:36.177129 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-7e293e16-c440-453a-a851-dd561ed8ca58 req-0cb95b6a-4bf1-4647-9c86-8b696aaface6 tempest-ServerActionsTestJSON-226995111 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:41:36.204346 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-7e293e16-c440-453a-a851-dd561ed8ca58 req-0cb95b6a-4bf1-4647-9c86-8b696aaface6 tempest-ServerActionsTestJSON-226995111 None] Connection info returned from driver {'name': 'volumes/volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '3e165d6c-bd44-4a15-909e-54cce95936fb'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:41:36.255414 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-7e293e16-c440-453a-a851-dd561ed8ca58 req-0cb95b6a-4bf1-4647-9c86-8b696aaface6 tempest-ServerActionsTestJSON-226995111 None] attachment_update completed successfully. Sep 17 21:41:42.239706 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-996d7b8c-6064-46ea-813c-fb3572e40761 req-e33dcdf7-02e0-4581-8e60-9b3b03db5c95 tempest-ServerActionsTestJSON-226995111 None] Terminate volume connection completed successfully. Sep 17 21:41:42.240289 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-996d7b8c-6064-46ea-813c-fb3572e40761 req-e33dcdf7-02e0-4581-8e60-9b3b03db5c95 tempest-ServerActionsTestJSON-226995111 None] Deleting attachment 3e165d6c-bd44-4a15-909e-54cce95936fb. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:41:43.424609 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] Acquiring lock "cinder-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:41:43.435364 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] Lock "cinder-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:41:43.469391 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:41:43.510301 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:41:43.510749 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] Volume volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:41:43.514867 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] deleting rbd volume volume-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:41:43.704039 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] Created reservations ['647c47ca-c5e3-48dd-ac98-18ebfd1545aa', '5ed9ce63-dcd0-45ba-8933-09f903ceed6f', '79506621-861d-4ea7-9b97-9edca9375ec5', '349dbb41-c0f5-4804-ad1b-f10c67f93077'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:41:43.750443 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:41:43.776151 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:41:43.802141 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:41:43.806573 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] Deleted volume successfully. Sep 17 21:41:43.819307 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-9a7c83b5-603e-4f5a-bee0-0330c511b471 tempest-ServerActionsTestJSON-226995111 None] Lock "cinder-1b39e8a5-9ba7-4ac0-bf9f-2a0c1bcbad0b-delete_volume" released by "delete_volume" :: held 0.384s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:42:01.027153 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:42:01.027923 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:42:01.057167 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:42:01.083296 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:42:04.918693 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=9,cinder:SELECT=36,cinder:INSERT=2,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:04.919838 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=41,cinder:UPDATE=16,cinder:INSERT=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:24.928250 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:44.925241 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=9,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:42:54.928271 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:43:01.028966 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:43:01.029782 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:43:01.095001 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:43:01.131470 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:43:04.934369 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:43:23.309745 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:43:23.328919 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Flow 'volume_create_manager' (2be07cc0-27bb-46e8-9f28-59344ef82ee8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:43:23.335196 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9ca80439-da8d-42a9-a84d-c3002b5f428e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:43:23.352977 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9ca80439-da8d-42a9-a84d-c3002b5f428e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:43:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-859792793',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f4da1dea-5959-4df8-a95c-887273815e5f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e3015a88384447ce8ee07537d685f3a3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:43:23Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:43:23Z,use_quota=True,user_id='806819c3594d48fe9bf3d0cce5fee164',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:43:23.358024 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d2674a00-8db1-4f6f-a61b-0f1c1db5a42a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:43:23.361994 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d2674a00-8db1-4f6f-a61b-0f1c1db5a42a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:43:23.366777 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (54684357-c3b0-494d-b207-a0a66accf70c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:43:23.370662 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (54684357-c3b0-494d-b207-a0a66accf70c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f4da1dea-5959-4df8-a95c-887273815e5f', 'volume_name': 'volume-f4da1dea-5959-4df8-a95c-887273815e5f', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:43:23.375293 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (60e1b5d1-bb45-4e3b-9da6-d5b05c2f2a5e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:43:23.379072 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (60e1b5d1-bb45-4e3b-9da6-d5b05c2f2a5e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:43:23.383422 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5dca3466-09a8-4be2-b5d5-9ac0193d8c58) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:43:23.384432 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Volume f4da1dea-5959-4df8-a95c-887273815e5f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f4da1dea-5959-4df8-a95c-887273815e5f', 'volume_size': 1} Sep 17 21:43:23.384817 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] creating volume 'volume-f4da1dea-5959-4df8-a95c-887273815e5f' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:43:23.385604 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:43:23.504332 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:43:23.504618 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:43:23.508235 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5dca3466-09a8-4be2-b5d5-9ac0193d8c58) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f4da1dea-5959-4df8-a95c-887273815e5f', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:43:23.512688 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1a61cc47-91ff-4101-b077-c50719635521) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:43:23.523673 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Volume volume-f4da1dea-5959-4df8-a95c-887273815e5f (f4da1dea-5959-4df8-a95c-887273815e5f): created successfully Sep 17 21:43:23.527480 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1a61cc47-91ff-4101-b077-c50719635521) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:43:23.532751 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Flow 'volume_create_manager' (2be07cc0-27bb-46e8-9f28-59344ef82ee8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:43:23.541785 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-e9c2c29c-58b0-4552-ae57-c7a4021e4d3f tempest-TestMinimumBasicScenario-231644504 None] Created volume successfully. Sep 17 21:43:24.941358 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=9,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:43:24.966953 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-b60e7023-f841-4976-8364-0f7ca4a0954e req-bc7c2a42-6017-4cb9-9783-ab5cd1b3bad3 tempest-TestMinimumBasicScenario-231644504 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:43:26.312984 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-b60e7023-f841-4976-8364-0f7ca4a0954e req-bc7c2a42-6017-4cb9-9783-ab5cd1b3bad3 tempest-TestMinimumBasicScenario-231644504 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.346s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:43:26.313841 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-b60e7023-f841-4976-8364-0f7ca4a0954e req-bc7c2a42-6017-4cb9-9783-ab5cd1b3bad3 tempest-TestMinimumBasicScenario-231644504 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f4da1dea-5959-4df8-a95c-887273815e5f', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': 'f4da1dea-5959-4df8-a95c-887273815e5f', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:43:26.340145 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-b60e7023-f841-4976-8364-0f7ca4a0954e req-bc7c2a42-6017-4cb9-9783-ab5cd1b3bad3 tempest-TestMinimumBasicScenario-231644504 None] Connection info returned from driver {'name': 'volumes/volume-f4da1dea-5959-4df8-a95c-887273815e5f', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'f4da1dea-5959-4df8-a95c-887273815e5f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '81d6a7dd-2d9b-4883-971b-a207a0e5fef6'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:43:26.394039 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-b60e7023-f841-4976-8364-0f7ca4a0954e req-bc7c2a42-6017-4cb9-9783-ab5cd1b3bad3 tempest-TestMinimumBasicScenario-231644504 None] attachment_update completed successfully. Sep 17 21:43:28.307792 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-c6af59f0-cc9b-4b17-97fa-400ccfa5ac4f tempest-TestMinimumBasicScenario-231644504 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:43:29.388880 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-c6af59f0-cc9b-4b17-97fa-400ccfa5ac4f tempest-TestMinimumBasicScenario-231644504 None] Create snapshot completed successfully Sep 17 21:43:44.934221 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=20,cinder:UPDATE=2,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:43:44.937527 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=24,cinder:UPDATE=7,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:44:01.026934 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:44:01.028056 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:44:01.063290 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:44:01.087679 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:44:14.945989 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:44:14.947206 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:44:54.946475 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:44:54.951515 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=9,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:45:01.024720 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:45:01.025829 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:45:01.056901 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:45:01.081063 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:45:14.956513 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:45:18.085791 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b2bf583c-65bc-4da2-93de-0345a56c6373 tempest-TestMinimumBasicScenario-231644504 None] Acquiring lock "cinder-d8b2c964-5dff-4a1b-8326-9877b6b02697-delete_snapshot" by "delete_snapshot" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:45:18.095713 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b2bf583c-65bc-4da2-93de-0345a56c6373 tempest-TestMinimumBasicScenario-231644504 None] Lock "cinder-d8b2c964-5dff-4a1b-8326-9877b6b02697-delete_snapshot" acquired by "delete_snapshot" :: waited 0.010s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:45:18.097278 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-b2bf583c-65bc-4da2-93de-0345a56c6373 tempest-TestMinimumBasicScenario-231644504 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:45:34.953168 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:45:44.966011 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:45:48.242111 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-b2bf583c-65bc-4da2-93de-0345a56c6373 tempest-TestMinimumBasicScenario-231644504 None] Created reservations ['00d0c310-79c1-49e8-bbf8-260092cf30bc', '236d2f55-9347-4886-bfe1-0122e4e89545', '3589d596-8224-48e7-8678-d919acf56c5a', 'fb03ecda-7e79-4041-bc3c-c16fb38b17c7'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:45:48.280510 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-b2bf583c-65bc-4da2-93de-0345a56c6373 tempest-TestMinimumBasicScenario-231644504 None] Delete snapshot completed successfully. Sep 17 21:45:48.285121 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-b2bf583c-65bc-4da2-93de-0345a56c6373 tempest-TestMinimumBasicScenario-231644504 None] Lock "cinder-d8b2c964-5dff-4a1b-8326-9877b6b02697-delete_snapshot" released by "delete_snapshot" :: held 30.189s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:46:01.026764 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:46:01.028441 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:46:01.063714 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:46:01.088105 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:46:14.960096 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=13,cinder:INSERT=3,cinder:UPDATE=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:46:14.961205 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=15,cinder:UPDATE=10,cinder:INSERT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:46:34.962494 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:46:54.956424 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=9,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:47:01.024855 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:47:01.025969 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:47:01.056997 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:47:01.081568 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:47:07.266329 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:47:07.278157 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-443a91e7-488b-46b9-80e0-40c9a5fb078d req-58ea51dc-9bf8-4cd1-8202-0aa9efb36cf6 tempest-TestMinimumBasicScenario-231644504 None] Terminate volume connection completed successfully. Sep 17 21:47:07.278571 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-443a91e7-488b-46b9-80e0-40c9a5fb078d req-58ea51dc-9bf8-4cd1-8202-0aa9efb36cf6 tempest-TestMinimumBasicScenario-231644504 None] Deleting attachment 81d6a7dd-2d9b-4883-971b-a207a0e5fef6. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:47:08.409143 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] Acquiring lock "cinder-f4da1dea-5959-4df8-a95c-887273815e5f-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:47:08.418526 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] Lock "cinder-f4da1dea-5959-4df8-a95c-887273815e5f-delete_volume" acquired by "delete_volume" :: waited 0.009s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:47:08.454067 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:47:08.493700 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:47:08.494206 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] Volume volume-f4da1dea-5959-4df8-a95c-887273815e5f is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:47:08.498489 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] deleting rbd volume volume-f4da1dea-5959-4df8-a95c-887273815e5f {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:47:08.686062 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] Created reservations ['d264d3db-9da9-4085-a163-dab99a2c0bbd', 'a0fb4a2a-5cfc-4931-a04c-7f84480e5207', 'b4661400-af34-438e-b1c8-e399573effd4', '76afd83e-787d-429d-9b3c-5315a4c43cae'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:47:08.731766 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:47:08.755372 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:47:08.779619 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:47:08.784257 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] Deleted volume successfully. Sep 17 21:47:08.796578 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-40487c42-806d-4fbe-95d0-467ae4bc4dc0 tempest-TestMinimumBasicScenario-231644504 None] Lock "cinder-f4da1dea-5959-4df8-a95c-887273815e5f-delete_volume" released by "delete_volume" :: held 0.378s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:47:18.789417 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=14,cinder:UPDATE=11,cinder:INSERT=2,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:47:34.971568 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=18,cinder:INSERT=2,cinder:UPDATE=8 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:47:42.966849 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 21:47:42.986272 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Flow 'volume_create_manager' (87726092-d3d4-461c-88e1-2fa3825f8511) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:47:42.992524 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0686e286-e84c-4f94-93d7-4703567da85f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:47:43.009033 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0686e286-e84c-4f94-93d7-4703567da85f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T21:47:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-2089899683',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=20302c04-a690-4750-9ebe-75495dc664c0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e3015a88384447ce8ee07537d685f3a3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T21:47:43Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T21:47:43Z,use_quota=True,user_id='806819c3594d48fe9bf3d0cce5fee164',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:47:43.014189 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (76a16c88-8a44-40b0-98c7-6be5ed539c4f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:47:43.018253 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (76a16c88-8a44-40b0-98c7-6be5ed539c4f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:47:43.023182 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (063969a3-3034-48b6-a319-e37682ee9f13) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:47:43.027292 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (063969a3-3034-48b6-a319-e37682ee9f13) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '20302c04-a690-4750-9ebe-75495dc664c0', 'volume_name': 'volume-20302c04-a690-4750-9ebe-75495dc664c0', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:47:43.032253 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7503bc68-95e7-40f9-9380-ecd242b4c09b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:47:43.035971 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7503bc68-95e7-40f9-9380-ecd242b4c09b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:47:43.040443 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0a92f2f9-5e5f-45f6-a929-20ae5ee4288f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:47:43.041532 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Volume 20302c04-a690-4750-9ebe-75495dc664c0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-20302c04-a690-4750-9ebe-75495dc664c0', 'volume_size': 1} Sep 17 21:47:43.041921 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] creating volume 'volume-20302c04-a690-4750-9ebe-75495dc664c0' {{(pid=133409) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1130}} Sep 17 21:47:43.042704 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:47:43.122672 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 21:47:43.123046 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 21:47:43.126866 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0a92f2f9-5e5f-45f6-a929-20ae5ee4288f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-20302c04-a690-4750-9ebe-75495dc664c0', 'volume_size': 1}' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:47:43.131340 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ef81c32a-0ee5-4741-a68b-7f0217ad44a1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 21:47:43.141960 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Volume volume-20302c04-a690-4750-9ebe-75495dc664c0 (20302c04-a690-4750-9ebe-75495dc664c0): created successfully Sep 17 21:47:43.145337 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ef81c32a-0ee5-4741-a68b-7f0217ad44a1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 21:47:43.150657 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Flow 'volume_create_manager' (87726092-d3d4-461c-88e1-2fa3825f8511) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 21:47:43.159999 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-4f55cf8b-1446-4cf7-a3f8-0b4dbcd2adbb tempest-TestMinimumBasicScenario-231644504 None] Created volume successfully. Sep 17 21:47:44.581227 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-59cc6852-ba0b-4e91-b6fb-da60124001bf req-ce4ea1b8-45b5-4e42-b3d0-45316116c3f7 tempest-TestMinimumBasicScenario-231644504 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 21:47:45.389346 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-59cc6852-ba0b-4e91-b6fb-da60124001bf req-ce4ea1b8-45b5-4e42-b3d0-45316116c3f7 tempest-TestMinimumBasicScenario-231644504 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.808s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 21:47:45.390108 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-59cc6852-ba0b-4e91-b6fb-da60124001bf req-ce4ea1b8-45b5-4e42-b3d0-45316116c3f7 tempest-TestMinimumBasicScenario-231644504 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-20302c04-a690-4750-9ebe-75495dc664c0', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '20302c04-a690-4750-9ebe-75495dc664c0', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 21:47:45.414212 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-59cc6852-ba0b-4e91-b6fb-da60124001bf req-ce4ea1b8-45b5-4e42-b3d0-45316116c3f7 tempest-TestMinimumBasicScenario-231644504 None] Connection info returned from driver {'name': 'volumes/volume-20302c04-a690-4750-9ebe-75495dc664c0', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '20302c04-a690-4750-9ebe-75495dc664c0', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'cfa8cafd-162b-4f5c-b57d-cb7b5c0c8abb'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 21:47:45.470206 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-59cc6852-ba0b-4e91-b6fb-da60124001bf req-ce4ea1b8-45b5-4e42-b3d0-45316116c3f7 tempest-TestMinimumBasicScenario-231644504 None] attachment_update completed successfully. Sep 17 21:47:54.969974 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=20,cinder:UPDATE=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:48:01.027963 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:48:01.029291 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:48:01.066413 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:48:01.091320 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:48:14.967578 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=23,cinder:UPDATE=6,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:48:34.970590 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:48:34.971458 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:48:44.998085 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:49:00.519008 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-7661816e-5d81-4eee-91d8-31530d7a6a94 req-bde8cf57-88d9-498a-88cb-91920cd4a1e4 tempest-TestMinimumBasicScenario-231644504 None] Terminate volume connection completed successfully. Sep 17 21:49:00.519562 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-7661816e-5d81-4eee-91d8-31530d7a6a94 req-bde8cf57-88d9-498a-88cb-91920cd4a1e4 tempest-TestMinimumBasicScenario-231644504 None] Deleting attachment cfa8cafd-162b-4f5c-b57d-cb7b5c0c8abb. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 21:49:01.026248 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:49:01.027182 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:49:01.060216 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:49:01.099877 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:49:01.159243 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] Acquiring lock "cinder-20302c04-a690-4750-9ebe-75495dc664c0-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 21:49:01.172485 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] Lock "cinder-20302c04-a690-4750-9ebe-75495dc664c0-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 21:49:01.204046 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:49:01.242193 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 21:49:01.242655 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] Volume volume-20302c04-a690-4750-9ebe-75495dc664c0 is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 21:49:01.247133 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] deleting rbd volume volume-20302c04-a690-4750-9ebe-75495dc664c0 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 21:49:01.431882 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] Created reservations ['6cc493f3-3dfd-4d22-a607-3622d3bfeab4', '296f657e-14d3-42c9-b446-025303d10147', '817a4ab1-52b9-4de1-b09b-0d0f52cd57bb', 'e40d4506-2fab-4556-a343-e51cd90a6345'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 21:49:01.488796 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:49:01.513663 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:49:01.538955 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:49:01.543275 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] Deleted volume successfully. Sep 17 21:49:01.560014 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-0941c59b-a0ea-47eb-84ce-cb4861afc6c2 tempest-TestMinimumBasicScenario-231644504 None] Lock "cinder-20302c04-a690-4750-9ebe-75495dc664c0-delete_volume" released by "delete_volume" :: held 0.387s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 21:49:14.970939 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=21,cinder:UPDATE=8,cinder:INSERT=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:49:24.981880 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=20,cinder:UPDATE=12,cinder:INSERT=1,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:49:44.989437 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:50:01.026219 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:50:01.027373 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:50:01.062320 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:50:01.087234 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:50:04.977943 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:50:24.974554 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:50:44.972806 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:51:01.024163 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:51:01.025310 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:51:01.062546 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:51:01.093447 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:51:04.971317 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:51:24.981362 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:51:34.988387 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:52:01.028712 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:52:01.029586 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:52:01.060847 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:52:01.086703 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:52:04.981102 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=2,cinder:SELECT=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:52:14.982923 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:52:34.981035 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:52:44.980915 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:53:01.032653 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:53:01.033380 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:53:01.065699 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:53:01.090798 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:53:04.984194 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:53:24.983425 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:53:44.988346 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:54:01.037009 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:54:01.037761 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:54:01.089793 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:54:01.124912 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:54:04.995355 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:54:24.998629 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:54:25.002726 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:54:55.000796 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:55:01.039987 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:55:01.040739 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:55:01.067291 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:55:01.098683 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:55:05.000146 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=10,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:55:35.006652 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:55:44.998794 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:01.045263 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:56:01.046010 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:56:01.074895 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:56:01.101694 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:56:15.010615 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=1,cinder:SELECT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:15.013611 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:45.012868 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:56:45.033971 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:57:01.046944 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:57:01.047697 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:57:01.075286 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:57:01.101056 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:57:05.023263 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:57:25.025434 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:57:25.030689 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=4,cinder:SELECT=6 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:57:45.043379 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:57:55.038337 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:58:01.047130 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:58:01.048090 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:58:01.081983 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:58:01.105264 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:58:25.037816 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:58:25.048094 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:58:45.053977 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:58:55.056937 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:01.051736 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 21:59:01.052758 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:59:01.086661 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 21:59:01.110263 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 21:59:15.060199 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:15.060880 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:45.063972 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 21:59:45.068018 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:00:01.056986 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:00:01.058162 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:00:01.096084 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:00:01.120748 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:00:25.065370 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=3,cinder:SELECT=4 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:00:25.066920 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:00:55.064723 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:01:01.057992 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:01:01.059363 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:01:01.092781 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:01:01.121197 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:01:02.590106 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=133409) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}} Sep 17 22:01:02.609121 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Flow 'volume_create_manager' (4bbbc106-7184-4497-b952-2eb44b2f2a8e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 22:01:02.615285 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ec1d494f-a606-495f-a61c-35d65bf1ecc9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 22:01:02.619508 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:01:02.630679 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ec1d494f-a606-495f-a61c-35d65bf1ecc9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-09-17T22:01:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-938828875',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='015ec23d11f64d56844d28795ccba61a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-09-17T22:01:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-09-17T22:01:03Z,use_quota=True,user_id='3d3bf2dacad34c0aa465968f1c273cf9',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(c8f1030c-5609-46eb-ad8a-9bbb096c5617),volume_type_id=c8f1030c-5609-46eb-ad8a-9bbb096c5617)' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 22:01:02.635758 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51de7e84-7107-4406-9ba0-28819637fb33) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 22:01:02.639799 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (51de7e84-7107-4406-9ba0-28819637fb33) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 22:01:02.644613 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2fecefb3-856c-42ea-a595-375d05bbd98c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 22:01:02.733049 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (2fecefb3-856c-42ea-a595-375d05bbd98c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'volume_name': 'volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 22:01:02.737992 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a5d8a86a-9ec7-4eab-aeb5-f8daaab323b0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 22:01:02.741784 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a5d8a86a-9ec7-4eab-aeb5-f8daaab323b0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 22:01:02.746409 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1641ab3a-ea53-4516-8efc-bdb8a301a69e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 22:01:02.747486 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Volume 7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } Sep 17 22:01:02.748054 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Cloning 7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43 from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 at location (None, None). {{(pid=133409) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}} Sep 17 22:01:02.748811 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Acquiring lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" by "_prepare_image_cache_entry" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 22:01:02.769746 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" acquired by "_prepare_image_cache_entry" :: waited 0.021s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 22:01:02.782507 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 22, 1, 2, 774602)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 22:01:02.782943 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 22:01:02.783555 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Found cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}} Sep 17 22:01:02.787700 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Lock "cinder-1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4" released by "_prepare_image_cache_entry" :: held 0.018s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 22:01:02.788596 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Attempting to retrieve cache entry for image = 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 on host ncdv-master-1@ceph#ceph. {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}} Sep 17 22:01:02.794529 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Found image-volume cache entry: {'id': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'volume_id': '79c922c8-a994-4aac-93a5-981c78ffc3d9', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34), 'last_used': datetime.datetime(2022, 9, 17, 22, 1, 2, 792111)}. {{(pid=133409) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} Sep 17 22:01:02.794942 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.image.cache [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Image-volume cache entry image_update_at = 2022-09-17 20:41:34+00:00, requested image updated_at = 2022-09-17 20:41:34+00:00. {{(pid=133409) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} Sep 17 22:01:02.795357 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Creating from source image-volume 79c922c8-a994-4aac-93a5-981c78ffc3d9 {{(pid=133409) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}} Sep 17 22:01:02.808866 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:01:02.862132 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] creating snapshot='volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43.clone_snap' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:846}} Sep 17 22:01:03.135188 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] cloning 'volume-79c922c8-a994-4aac-93a5-981c78ffc3d9@volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43.clone_snap' to 'volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43' {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:852}} Sep 17 22:01:03.256663 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Volume volume-d32359bf-ad08-4517-9759-3f42907f23c0.deleted is not a clone. {{(pid=133409) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1291}} Sep 17 22:01:03.257836 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:982}} Sep 17 22:01:03.258212 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=133409) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:991}} Sep 17 22:01:03.274991 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] clone created successfully {{(pid=133409) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:925}} Sep 17 22:01:03.275643 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.volume_utils [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Marking volume 7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43 as bootable. {{(pid=133409) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1140}} Sep 17 22:01:03.285959 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Copying metadata from image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 to 7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43. {{(pid=133409) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}} Sep 17 22:01:03.286416 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.flows.manager.create_volume [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Creating volume glance metadata for volume 7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43 backed by image 1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=133409) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}} Sep 17 22:01:03.338854 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1641ab3a-ea53-4516-8efc-bdb8a301a69e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'volume_size': 1, 'image_id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'c8633d9055db49bfa42074c66a0581c2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4', 'created_at': datetime.datetime(2022, 9, 17, 20, 41, 18, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 9, 17, 20, 41, 34, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/1d08c0bf-f5b9-44bb-9eaf-23051e1d70a4/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 22:01:03.343452 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7da5383d-7b08-470c-a055-f8e786f54fdf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} Sep 17 22:01:03.353728 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.flows.manager.create_volume [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Volume volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43 (7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43): created successfully Sep 17 22:01:03.357152 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7da5383d-7b08-470c-a055-f8e786f54fdf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=133409) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} Sep 17 22:01:03.362459 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Flow 'volume_create_manager' (4bbbc106-7184-4497-b952-2eb44b2f2a8e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=133409) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} Sep 17 22:01:03.372480 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [None req-954b4ba6-4fdb-4478-a100-c17c59777892 tempest-TestShelveInstance-1018269517 None] Created volume successfully. Sep 17 22:01:07.655546 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-a1b7630b-ebe7-4de5-ae06-680c87a4d469 req-d880c4e8-f93f-41ca-be22-8fbff703eca0 tempest-TestShelveInstance-1018269517 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 22:01:08.507597 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-a1b7630b-ebe7-4de5-ae06-680c87a4d469 req-d880c4e8-f93f-41ca-be22-8fbff703eca0 tempest-TestShelveInstance-1018269517 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.852s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 22:01:08.508731 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-a1b7630b-ebe7-4de5-ae06-680c87a4d469 req-d880c4e8-f93f-41ca-be22-8fbff703eca0 tempest-TestShelveInstance-1018269517 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 22:01:08.533943 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-a1b7630b-ebe7-4de5-ae06-680c87a4d469 req-d880c4e8-f93f-41ca-be22-8fbff703eca0 tempest-TestShelveInstance-1018269517 None] Connection info returned from driver {'name': 'volumes/volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '8d082cbd-9e11-4ead-a23e-62918a79c8e0'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 22:01:08.587283 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-a1b7630b-ebe7-4de5-ae06-680c87a4d469 req-d880c4e8-f93f-41ca-be22-8fbff703eca0 tempest-TestShelveInstance-1018269517 None] attachment_update completed successfully. Sep 17 22:01:35.069093 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=31,cinder:INSERT=6,cinder:UPDATE=4,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:01:35.070957 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=30,cinder:UPDATE=8,cinder:INSERT=6 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:02:01.056418 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:02:01.058131 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:02:01.095008 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:02:01.119965 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:02:15.075790 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=9,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:02:15.077502 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=3,cinder:SELECT=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:02:55.069974 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:02:55.071518 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:03:01.056957 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:03:01.058334 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:03:01.088567 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:03:01.113375 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:03:35.071991 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:03:35.078817 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:03:37.634014 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-84d2ce87-c3f3-41b7-a802-0f90c0fcf8ca req-a7f0260f-150a-4c48-9539-88a467392eb2 tempest-TestShelveInstance-1018269517 None] Terminate volume connection completed successfully. Sep 17 22:03:37.634741 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-84d2ce87-c3f3-41b7-a802-0f90c0fcf8ca req-a7f0260f-150a-4c48-9539-88a467392eb2 tempest-TestShelveInstance-1018269517 None] Deleting attachment 8d082cbd-9e11-4ead-a23e-62918a79c8e0. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 22:03:39.773072 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-1021df92-b3ef-453a-8279-168a324d9a83 req-4a36e40c-3f54-43b4-9341-a0c4b684df4e tempest-TestShelveInstance-1018269517 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 17 22:03:40.508628 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_concurrency.processutils [req-1021df92-b3ef-453a-8279-168a324d9a83 req-4a36e40c-3f54-43b4-9341-a0c4b684df4e tempest-TestShelveInstance-1018269517 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.735s {{(pid=133409) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 17 22:03:40.509497 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-1021df92-b3ef-453a-8279-168a324d9a83 req-4a36e40c-3f54-43b4-9341-a0c4b684df4e tempest-TestShelveInstance-1018269517 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '4391210e-4b15-4c1d-b970-cae3efa14448', 'volume_id': '7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'discard': True}} {{(pid=133409) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1779}} Sep 17 22:03:40.536791 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-1021df92-b3ef-453a-8279-168a324d9a83 req-4a36e40c-3f54-43b4-9341-a0c4b684df4e tempest-TestShelveInstance-1018269517 None] Connection info returned from driver {'name': 'volumes/volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'hosts': ['10.222.0.26'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'fb5226a7-bf9f-4a2d-a664-3514707a329b'} {{(pid=133409) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4867}} Sep 17 22:03:40.590543 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-1021df92-b3ef-453a-8279-168a324d9a83 req-4a36e40c-3f54-43b4-9341-a0c4b684df4e tempest-TestShelveInstance-1018269517 None] attachment_update completed successfully. Sep 17 22:04:01.059839 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:04:01.061199 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:04:01.093881 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:04:01.118305 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:04:05.078207 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=15,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:04:15.075343 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=21,cinder:UPDATE=6 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:04:35.072382 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:04:45.081252 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:01.064691 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:05:01.065852 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:05:01.103692 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:05:01.127442 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:05:05.085965 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:19.624356 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:19.634427 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-ae9dc75f-2cb5-44e8-ac80-18be6a969f0f tempest-TestShelveInstance-1018269517 None] Terminate volume connection completed successfully. Sep 17 22:05:19.634839 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.manager [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-ae9dc75f-2cb5-44e8-ac80-18be6a969f0f tempest-TestShelveInstance-1018269517 None] Deleting attachment fb5226a7-bf9f-4a2d-a664-3514707a329b. {{(pid=133409) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4998}} Sep 17 22:05:19.762480 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] Acquiring lock "cinder-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43-delete_volume" by "delete_volume" {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:192}} Sep 17 22:05:19.773267 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] Lock "cinder-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43-delete_volume" acquired by "delete_volume" :: waited 0.011s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:196}} Sep 17 22:05:19.808359 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:05:19.896726 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] volume has no backup snaps {{(pid=133409) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1263}} Sep 17 22:05:19.905187 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] deleting rbd volume volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43 {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1415}} Sep 17 22:05:20.107816 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] volume is a clone so cleaning references {{(pid=133409) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1436}} Sep 17 22:05:20.162204 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] deleting parent snapshot volume-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43.clone_snap {{(pid=133409) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1329}} Sep 17 22:05:21.259422 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.quota [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] Created reservations ['9b862f0f-19a8-457d-8a38-d2ac0cbb5091', '45f7a2e1-ccfd-4203-8f2e-f47a0df00d20', 'ffd2aabd-8e7d-49c5-aec7-2e823301f8a8', 'c2d4129a-6b5c-4e3a-82f2-cc1b46fc4f91'] {{(pid=133409) reserve /opt/stack/cinder/cinder/quota.py:755}} Sep 17 22:05:21.307520 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:05:21.343233 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:05:21.375705 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:05:21.382591 ncdv-master-1 cinder-volume[133409]: INFO cinder.volume.manager [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] Deleted volume successfully. Sep 17 22:05:21.399686 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.coordination [req-25db84db-ee9b-41ff-bb5d-6af54f412398 req-68a94343-fef4-48da-924e-1c2041041956 tempest-TestShelveInstance-1018269517 None] Lock "cinder-7f79dedc-7a7c-44ae-8f34-f9ddfb2ddb43-delete_volume" released by "delete_volume" :: held 1.626s {{(pid=133409) _synchronized /opt/stack/cinder/cinder/coordination.py:207}} Sep 17 22:05:35.084848 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=13,cinder:INSERT=3,cinder:UPDATE=10 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:05:45.083232 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=23,cinder:UPDATE=9,cinder:INSERT=1,cinder:DELETE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:06:01.063869 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:06:01.064823 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:06:01.097241 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:06:01.121508 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:06:15.089926 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:06:15.090980 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:06:35.091083 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:06:45.094573 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:07:01.068754 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:07:01.070089 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:07:01.104802 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:07:01.130901 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:07:05.106700 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:07:15.109282 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:07:45.107169 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:07:55.113788 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:08:01.067420 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:08:01.069414 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:08:01.098868 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:08:01.136546 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:08:15.119039 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:08:25.121553 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:08:35.127400 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:08:55.133419 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:09:01.069927 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:09:01.071535 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:09:01.103230 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:09:01.127807 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:09:15.128478 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:09:35.125857 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:09:55.129805 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:10:01.073683 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:10:01.074378 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:10:01.110226 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:10:01.146167 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:10:15.123397 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:UPDATE=3,cinder:SELECT=5 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:10:35.131103 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:10:45.139168 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:11:01.072928 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:11:01.074525 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:11:01.106004 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:11:01.130323 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:11:15.131003 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:11:25.138801 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:11:55.129827 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:12:01.077088 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:12:01.078209 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:12:01.111926 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:12:01.146222 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:12:05.134501 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:12:25.143454 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:12:45.134031 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:13:01.074511 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:13:01.075525 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:13:01.107340 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:13:01.131368 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:13:05.143219 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:13:15.149946 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:13:45.135548 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:13:55.147168 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:14:01.078154 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:14:01.078803 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:14:01.108153 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:14:01.131842 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:14:15.137753 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:14:35.148242 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:14:45.159558 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:15:01.078285 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:15:01.079296 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:15:01.118873 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:15:01.147181 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:15:05.162473 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:15:05.163417 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:15:25.164314 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:15:45.163321 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:16:01.084610 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:16:01.085690 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:16:01.120761 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:16:01.144640 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:16:05.166311 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:16:25.160809 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:16:45.164328 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:17:01.089965 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:17:01.090971 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:17:01.122681 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:17:01.147640 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:17:05.176305 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=3 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:17:15.177851 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:17:45.174779 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:17:55.173416 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:18:01.092833 ncdv-master-1 cinder-volume[133409]: DEBUG oslo_service.periodic_task [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=133409) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} Sep 17 22:18:01.093968 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:18:01.124663 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.volume.drivers.rbd [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=133409) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:586}} Sep 17 22:18:01.148929 ncdv-master-1 cinder-volume[133409]: DEBUG cinder.manager [None req-3a8b6692-1139-42cf-8714-71e7cbe69a2a None None] Notifying Schedulers of capabilities ... {{(pid=133409) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} Sep 17 22:18:15.184155 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 17 22:18:15.185227 ncdv-master-1 cinder-volume[133409]: DEBUG dbcounter [-] [133409] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=133409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}